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ABSTRACT 

This volume serves as an introduction to and a 
summary report of the project and provides a description of the 
context, the rationale, and the objectives of the Studies. The first 
five chapters describe the purposes to be served by the study, the 
preparations necessary to serve those purposes, the directions taken 
and the decisions made to accomplish the work, the variables selected 
as sources for data collection, and the primary vehicle used to 
present these data. Three of the next seven chapters serve as outline 
maps descriptive of the personnel, the outputs, and the work 
requirements found in educational RDDSE. The remaining four data 
chapters represent detailed maps, three of which describe the data in 
terms of the outputs selected as the focus of data collection, and 
one of which deals with special analyses of the data in response to a 
limited set of specific questions. The final four chapters of the 
volume discuss the implications of the Oregon Studies data for (1) 
conceptual and methodological development, (2) training personnel, 

(3) developing training models, and (4) further exploration and 
study. Related Documents are EA 004 583-589. (Pages 394 and 396 may 
reproduce poorly.) (Author/JH) 
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AN OVERVIEW OF THE OREGON STUDIES IN EDUCATIONAL RDD&E 



In the spring of 1970 the Training Branch of the U.S. 
Office of Education, National Center for Educational 
Research and Development, announced a plan to effect 
change in the preparation of educational RDD&E person- 
nel. Two factors led to the announcement. The underlying 
factor was the rather dramatic emergence in the past decade 
of development, diffusion, and evaluation activities as 
vehicles for educational improvement, and the attending 
need for qualified personnel to carry them out. The 
precipitating factor, however, was evidence that in spite of 
an investment of approximately 30 million dollars by the 
Federal Government to help training programs become 
more responsive to the personnel needs created by these 
new activities, essentially the same number and kind of 
personnel were being prepared in 1970 as in I96S. 

The plan for change reflected a strategy that can best be 
described as “beginning at the beginning.’’ It incorporated 
three interrelated lines of activity: the creation of a 
conceptual and empirical base on which to build functional 
training programs; the design of more effective and efficient 
approaches to training; and the development of instruc- 
tional materials that reflect desired changes in both content 
and procedure. The propositions on which the plan rested 
were straightforward: (a) little was known at <ut edu- 

cational development, diffusion and evaluation activities, or 
how they related to educational research; (b) even less was 
known about the training of personnel to carry out such 
activities; and (c) until both of these conditions were 
remedied the likelihood of designing effective and efficient 
programs to prepare personnel to carry them out was slight. 
The plan as a whole was coordinated so that the various 
activities within it would be developed with sensitivity to 
each other, and so that they would come together in 
completed fashion at approximately the same point in time. 
(For additional details on the plan for change see Chapter 1 
in Volume I of the series reporting the Oregon Studies.) 

The Oregon Studies, carried out by the Teaching 
Research Division of the Oregon State System of Higher 
Education, were to contribute in a beginning way to the 
conceptual and empirical base called for in the plan. As 
such they were to produce five products: a collection of 
detailed “case study’’ descriptions of projects that illus- 
trated exemplary RDD&E activities within various edu- 
cational contexts; a reliable, economically feasible method- 
ology by which to collect the data needed to prepare the 
case studies; a conceptual system or framework for viewing 
the domain of educational RDD&E that could be used as a 
guide to the classes of data to be attended to in the case 
studies; cross-project analyses that highlighted the simi- 



larities and differences observed in the projects described, 
and that tested in rudimentary fashion the adequacy of the 
conceptual framework underlying those observations; and a 
compendium of the existing literature that pertained to 
cither the nature of or the interactions between activities 
labeled educational research, development, diffusion and 
evaluation. These products are reported in five volumes: 



Volume I. Summary Report (with Technical 
Appendices) 

Volume II. The Literature of Educational RDD&E 
Part One (Research, Evaluation, and 
Development) 

Part Two (Diffusion & Combinations of 
RDD&E) 

Volume III. Conceptual Frameworks for Viewing 
Educational RDD&E 



Volume IV. Profiles of Exemplary Projects in 
Educational RDD&E 
Part One (Research and Evaluation) 
Part Two (Development) 

Part Three (Diffusion) 

Volume V. A Methodology for the Study of 
Educational RDD&E 



Each volume in the series reporting the Studies has been 
designed to stand alone, but because each volume reports a 
different product, and each product can be understood 
fully only in relation to the other products, two “reefer’s 
guides” to the series have been prepared. The first involves 
brief summaries or abstracts of the contents of each of the 
five volumes in the series. These appear on the inside of the 
back cover of the volume, and are intended to serve as a 
guide or overview to the series as a whole. A more detailed 
guide is provided by Volume I. In addition to serving as a 
general summary of the Studies, it contains descriptions of 
the developmental histories of the products reported in the 
various volumes, the relationships that exist between them, 
and the runner in which they have interacted over time. 
Accordingly, for the reader who wishes to determine 
quickly what each of the five volumes in the series contains, 
turn to the inside of the back cover of the volume; for the 
reader who wishes to understand how the volumes relate to 
one another, follow that by reading Volume I. 
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ABSTRACT 



Mils; Is the first of five volumes reporting the results of t In* t'n >-on 
Studies in education.il research, development, diffusion and evaluation 
(educational RDD&E) . It contains 16 chapters which a) provide a hack 
ground to the study in terms of the questions attended to and the vari- 
ables studied, b) present the data gathered in the study, c) discus.*, in. 
plications of the data and d) set forth a set of recommendations. The. 
first five chapters of the volume contain information descriptive of the 
purposes to be served by the study, the preparations necessary to serve 
those purposes, the directions taken and the decisions made to accomplish 
the work, the variables selected as sources for data collection, and the 
primary vehicle used to present the data. The next seven chapters of 
the volume present various aspects of the data. Three data chapters 
serve as outline maps descriptive of the personnel, the outputs, and the 
work requirements found in educational RDD&E. The remaining four data 
chapters represent detailed maps, three of which describe the data in 
terms of the Outputs selected as the focus of data collection, and one of 
which deals with special analyses of the data in response to a limited 
set of specific questions. The final four chapters of the volume dis- 
cuss the implications of the Oregon Studies data for a) conceptual and 
methodological development, b) training personnel, c) developing train- 
ing models, and e) further exploration and study. 



PREFACE 



I lie* present volume serves as an introduction to and a summary report 
ol the Oregon Studies in educational research, developmert, diffusion, 
and evaluation (educational RDD&E) . As such it contains a description 
of the context within which the Studies were undertaken, the rationale 
for the Studies, and the objectives set for them. Jt also contains an 
overview of the methods and procedures used in the Studies, an analysis 
of the conceptual, methodological and training implications that follow 
from them, and recommendations for follow-on activities. Most critically, 
however, the volume reports in summary form the empirical data collected 
in the Oregon Studies on the nature and purpose of educational KDD&K. 

Two volumes In the series of volumes reporting the Oregon Studies 
contain empirical data. Volume IV contains a collection of "case pro- 
files" that describe 20 ongoing educational RDD&E projects. These 20 
profiles represent the primary outputs of the Oregon Studies, and con- 
tain all of the empirical data collected during the course of the Studies. 
The present volume contains a series of "cross project" analyses based 
on the data reported in the 20 case profiles. These analyses are ordered 
along a number of dimensions and provide, within the limits of the sample 
of projects described, data of high utility both for the theoretician 
and for the practitioner of educational RDD&E. 

By adopting a case study approach to the investigation of educational 
RDD&E activities, a wide range of data had to be collected. These data 
can be clustered within three broad classifications: general project 

descriptors; people descriptors; and work requirement descriptors. Gen- 
eral project descriptors which appear only in the case profiles, included 
such items as the objectives set for projects, organizational and manage- 
ment strategies used in projects, resource allocations, the political, 
intellectual and institutional contexts within which projects rest, and 
the "dynamics" of project operation. People descriptors, which appear 
in thd profiles and are summarized in Chapters 6 and 12 of the present 
volume, include such items as educational background and work experience, 
job roles and related work activities, and resources relied upon in the 
performance of job related activities. Such data are presented for 134 
of the 307 professional staff identified in the 20 projects studied, and 
were obtained from questionnaires administered as part of an onsite data 
collection process. 

The third broad class of data collected in the Oregon Studies deals 
with the outputs produced by educational RDD&E projects, and the nature 
of the work required to produce them. These are the core data generated 
by the Oregon Studies, and consist of four major data sets: (cO the 

outputs that emerge from educational RDD&E projects; (b) the standards 
held for those outputs; ( c ) the operations (activities, tasks, actions) 
involved in producing an output to its specified standards; and (d) the 
enabling knowledge : , skills and sensitivities needed to carry out those 
operations. Such data are presented for 298 of the 962 outputs identi- 
fied Ln the 20 projects studied, and were collected through onsite in- 
terviews with personnel responsible for the production of the outputs 
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sol nr tod aspects of the output * v.vrl- requirement data 
bar ic t e r i s t i t:s , for example, project si/e and instl- 
Im oni 1 * in.itiuns, these various analyses, supplemented 
■nnicai appendices, proviso a reasonably ». ompruhrns.i ve 
'iri. is produced within n!u.-at i onn 1 KPDM; projects, 
i nvc 1 v e u in its production, ,\ I 1 data . c t, s used in 
’ iro described in Chapter '* ol fhn present volume. 



'iiviT, the function that Volume I was to play within (lie series of 
volumes reporting t lie Oregon Studies, the utility it was hoped to have 
lor various audiences, and tire technical nature of that which was bein>* 
reported, the form and content of ihe volume became critical. After 
cons i de r i ng a number of alternative formats, we chose to model the volume 
alter Lae reporting form of an exploratory or expeditionary force. We 
chose this form fir a number of reasons: nearly everyone associated with 

the Studies saw them as being exploratory in nature; everyone saw them 
as being responsible lor describing the process of "mapping the domain" 
of educ.it Iona l RDDaK; and the reading public is reasonably familiar with 
the iarguage of such reports, thanks to persons such as Lewis and Clark, 
Jacques Cousteau, and the flight crews of the Apollo program. Accord- 
ingly, the present volume is organized into chapters that deal with such 
topics as The Mission and the Challenge; Preparations; The Course Charted; 
Site Maps; Outline Maps; Special Purpose Maps; and Recommendations to 
the Sponsoring Agency. All are topics that every expeditionary force 
has t:o contend with, and they are topics that provide a reasonably good 
format for reporting the work of the Oregon Studies. It is hoped that 
the language of exploration, mapping, and the like will prove to be 
useful to the readers of the volume as it did to those who were respon- 
sible for its preparation. 



As is the case with nearly any mapping effort, the Oregon Studies 
expedition was not the first to have entered the territory that was to 
be mapped, and the maps emerging from the expedition can not be thought 
of as definitive. Put in other terms, the Oregon Studies expedition 
did not blaze an original trail within the territory labeled educational 
RI)D&iv t and hopefully it will not be the last mapping expedition to be com- 
missioned. It should be seen, rather, as the first major expedition hav- 
ing the explicit charge of systematically mapping the domain of educa- 
tional RDD&E. As such, it was responsible for pulling together much of 
the fragmentary information that existed on the territory, weaving that 
information together with the observations made on the expedition into 
a systematic whole, and in so doing creating a sound basis on which others 
might enter or work in the territory for whatever purpose. As with all 
major exploratory efforts, the expedition built upon the recorded obser- 
vations of persons who had traveled the territory at an earlier time 
(the literature reported in Volume II of the series), and those who were 
most knowledgeable of the territory were asked to prepare detailed charts 
and preliminary maps as guides to the efforts of tne expedition along 
the way (the conceptual papers reported in Volume LI I). 
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'Jlie Oregon Studies expedition, as with .my other single mapping 
expedition, suffered two major limitations. Given the time, energy, 
resources, skills and hark pround information available to the Studies 
only a limited number of sites could be studied for purposes of map 
building, and within any site studied, only a limited number of the 
var aihlcs that could be studied were considered. Just as Lewis and Clark 
could not base their maps upon a detailed exploration of all the rivers, 
va , leys and mountain ranges in the northwest, and the Apollo crews could 
not base their maps upon a detailed exploration of all major surface areas 
of the moon, the Oregon Studies were in a position of having to prepare 
maps of the domain of educational RDD&K on the basis of incomplete data. 
I.ven though a great deal of rare was taken with respe.t to the sites 
selected for study (see Chapter 3), and a great deal of care was taken 
with respect to the selection of variables to be studied within sites 
(see Chapter 4), the data base on which maps were to be prepared was 
limited. The work of the expedition provides the first set of maps that 
exist for the territory of educational RDD&E as a whole, however, and 
regardless of how ciude or inaccurate they may ultimately turn out to 
he, that fact alone makes them of value. For those who wish to travel 
across the territory, work within it, or undertake the task of mapping 
it further, there now exists at least some good “first approximat ions’ 1 
as to what that territory is like. 

As all who have administered a large scale R,D,D, or E project know, 
its successful completion requires the effort and cooperation of a great 
many persons. Ordinarily the names of such persons would appear in a 
volume of this kind, for it is altogether fitting that they be recognized 
publicly. Because of the unusally large number of persons that have con- 
tributed to the Oregon Studies, however, and the restrictions in space 
that accompany the present volume, only a selected few will be recognized 
here. Full and proper recognition of the persons involved is provided 
in the prefaces to the ''flumes that report the Studies in detail. 



Staff and consultants within the National Center for 
Educational Research and Development, U.S. Office of 
Education : From the initial conception of the project 

to its completion Dr. John Egermeir, Dr. Paul Messier 
and Ms. Cora Beebe of the Office of Education staff, and 
Dr. John E. Hopkins of Indiana University, played a key 
role in its planning and implementation. As Director of 
the Division of Higher Education within NCERD, and as sucli 
responsible for the administration of the Training Branch 
of the U.S. Office of Education, Dr. Messier was immediately 
responsible for both the initiation and implementation of 
the program within which the Oregon Studies were initiated.! 



!Dr. Messier is now directing planning for the proposed National 
Foundation for Higher Education, and the Training Branch has been re- 
named the Research Training Branch and moved from the Division of Higher 
Education to the Division of Research and Development Resources, National 
Center for Educational Research and Development. Dr. KgermJur Ik now 
serving as the director o I t lie Research Training Branch, with Dr. Susan 
S. KLein assuming the tusk of project officer. 
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From Its earliest planning, however, the program at; a 
whole, as veil as the Oregon Studies within it, wjih the 
result of a remarkably productive interact ion between all 
four persons identified. Dr. Kgermier functioned initially 
as project officer and coordinator of all three program 
thrusts, Ms, Beebe served as the staff represeiUnt i ve 
from the Division of Planning and (Evaluation, and Dr. 
Hopkins acted as the substantive specialist and stand- 
ing consultant to the program. Singly and in combina- 
tion these four persons have contributed to the form 
and function of the Studies In countless ways. 



Continuing consultants to the O regt m Stud ies : in order 

that a broad range of experience in map building within 
the behavioral sciences be incorporated within the Studies, 
three persons from related disciplines were asked to serve 
as continuing consultants to them. in this regard, Dr. 
James Watson of the University of Wash ingtnn represented 
the point of view of social anthropologist s ; Dr. Harry L. 
Ammerman, formerly of the Human Resources Research Organ- 
ization, repress ited the point of view of applied psycho- 
logists concerned with job and task analysis; and Dr. 

Samuel A. Sieber of Columbia University represented the 
point of view of social psychologists. Following Dr. 
Amraerman * s move to the staff of the Oregon Studies, Dr. 
Norman J. Boyan, formerly Director of the Bureau of 
Research, U.S. Office of Education and now Dean of the 
Graduate School of Education, University of California, 
Santa Barbara, represented the point of view of the 
applied psychologist. 



Authors and critiquers of conceptual papers : The Oregon 

Studies were designed so that there would be a close inter- 
play between conceptual and empirical activities at all 
times. To help in this regard the authors of a series of 
conceptual papers prepared as part of the Studies, and 
some of the critiquers of those papers, were present each 
time that the plans for the empirical thrust of the Studies 
were reviewed formally. As a consequence, the contributions 
of these persons to the project were as much in the realm 
of design and methodology as they were in the realm of concep- 
tualization. Dr. Leslie J. Briggs of Florida State Uni- 
versity; Dr, Hendrik D. GiJeonse, Dean of the School of 
Education, University of Cincinnati and formerly head of 
NCERD's Planning and Evaluation Branch; and Drs . Gene V 
Glass and Blaine R. Worthen of the Laboratory for Educa- 
tional Research, University of Colorado, were the authors 
of the conceptual papers. The critiquers of the indivi- 
dual papers, who attended all of the review conferences, 
were Dr. Norman J, Boyan of the University of California, 

Santa Barbara; Dr. Paul D. Hood of the Far West Regional 
Educational Laboratory; and Dr. Glen Heathers of the 
Learning Research and Development Center, University of 
Pittsburgh . 

vi 



Dir ecto r s o f p rojects studie d: Obviously, the directors 

of the 20 projects described In the Oregon Studies played 
a critical role in the mapping venture, for without their 
participation no maps would have been prepared. A wPlng- 
ness to submit one's work at the operational level to p-tbli; 
scrutiny, a willingness to give up three or more days of 
prime work time to meet other than Job demands, and a will- 
ingness to release key staff for as much as a day and a 
half for other than project activities are matters not to 
be taken lightly. While an agreement on the part of a 
project director to take part in the Oregon Studies was 
accompanied by an assurance of anonymity, and thus the 
names of the project directors do not appear at any point 
In the volumes reporting the Studies, the reader needs to 
be aware that they were in some ways the major benefactors 
of the Studies. 



Teaching Research staff and adminst rat ion : With the magni- 

tude and diversity of the Oregon Studies, and the short 
time lines within which they were to be carried out, staff 
had to be organized into a series of '’task forces" or teams. 
In the present volume only the function and composition of 
those teams will be noted: the excellence of their work 

and the pleasure of their company can only be alluded to. 

The literature review team, coordinated by Ms. Bette 
Porter, consisted of Ms. Lee Green, Ms. Bernett Samples, 

Ms, Frances Spigai and Ms. Sherrill L. Whittemoie; the 
conceptual framework team, coordinated by Dr. H. D. Schalock 
and Mr. G. Roger Sell, consisted essentially of the entire 
collection staff; the sit^ selection team, coordinated by 
Dr. Dale Hamrens, consisted of Mr. Steve Anderson, Mr. 
Darrell Clukey and Dr. James Nord; the site visitation 
team, coordinated by Dr. H. D. Schalock, consisted of 
Dr. Ammerman, Dr. Hamreus and Mr. Gregory Thomas; the 
data collection, reduction and profile preparation team, 
coordinated by Dr. Ammerman and Mr. Thomas, consisted 
of Mr. Loring Carl, Mr. Clukey, Mr. Norman Crowhurst, Ms. 
Green, Mr. Herb Hill, Ms. Diane Jones, Dr. Rod Myers, 

Dr. Nord, Hr. Dean Pielstick, and Mr. Clark A. Smith; 
the data analysis, storage and retrieval team, coor- 
dinated by Mr. Thomas, consisted of Dr. Anmerman, Mr. 

Carl, Mr. Clukey, Mr. William Hickok and Mr. Smith; the 
secretarial and financial assistance team, coordinated 
alternately by Mr. Larry Horyna and Mr. Thomas, consisted 
on various occasions of the late Ms. Jeri Jenkins, Ms. 

Donna Coutts, Ms. Bea Lindahl, Ms. Bernett Samples and 
Ms. Irene Zimmerman; and the intraagency review team of 
Drs. James H. Beai^d, H. D. Fredericks, C. F. Paulson, and 
John Williamson. 




To all who gave so much of their time, their energy, and their export 
ence, my deepest appreciation. 
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Chapter 1 



THE MISSION AND THE CHALLENGE 



While exact or subtle meanings may be lacking, most Americans are 
reasonably familiar with the concepts of research, development, evalua- 
tion and diffusion. ^ School children, business men, scientis ts educa- 
tors, and homemakers tend to see research as a set of activities that 
leads to the extension of knowledge, development as a set of activities 
that leads to the creation of technology, evaluation as a set of activ- 
ities that leads to decisions as to the worth or effectiveness of some- 
thing, and diffusion as a set of activities that leads to the adoption 
or utilization of that which emerges from research or development or 
evaluation. As such, most people see RDD&E activities as a set of 
problem-solving tools that man has applied with a great deal of success 
in the past and that appear to be critical to the solution of the 
larger and more difficult problems that he faces in the future. 

One has only to point to the recent history of the aircraft and 
space industry, agriculture, medicine, communications, and the military, 
and trace the role of RDD&E within them, for some sense of the utility 
of such activities as problem-solving tools in service of particular 
ends. A comparable history, however, is difficult to trace in educa- 
tion. Research has still to demonstrate to most American educators 
its contribution to on-line school operations; systematic development 
and evaluation activities, while being increasingly embraced as major 
new support systems for the schools, have still to demonstrate their 
capacity to make a genuine difference in the effectiveness of schooling; 
and the planned, systematic diffusion of promising educational practices 
has yet to be demonstrated on more than an Experimental” basis. While 
it is possible to point to contributions that RDD&E have made to the 
practice of education, it is probably fair to say that these activities 
have not yet proved as useful as problem-solving tools within the con- 
text of education as they have within other fields of man’s endeavor. 

A number of factors have been hypothesized as contributing to this 
state of affairs, for example, the inappropriateness of education as a 
context within which to apply the tools of RDD&E, the lack of adequate 
funding for such activities, the incompetence of those who have at- 
tempted to apply them to the problems of education. Two alternative 
explanations seem preferable: the relatively brief history of RDD&E 

within the context of education, and the failure thus far for these 



1 Formal definitions of research, development, diffusion- evalua- 
tion and related terms will be found in the GLOSSARY that accompanies 
the volume. When the terms research, development, diffusion, and 
evaluation are referred to collectively, the acronym RDD&E is often 
used . 
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t no I s to be adapted to the particular demands ol education. A h recog- 
nized entities, with recognized bodies of literature, educational 
development, evaluation, and diffusion have existed lor Little more than 
a decade (Chase, 1070), Educational research, of course, has had a 
much longer history, and has accumulated a much larger body of litera- 
ture, but it, too, has commanded major public support for only the past 
dozen or so years. Until very recently, educational RDD&E activities 
have been relatively unknown entities to the vast majority of personnel 
manning the nation's schools, or, for that matter, personnel manning 
the colleges, universities, R and D centers, and administrative agen- 
cies responsible for the management of such activities. As a conse- 
quence, it is only now in a sense that it is reasonable to expect con- 
tributions to be forthcoming from educational RDD&E, for it is only 
now that these activities are beginning to be a functional aspect of 
the educational enterprise. 

The .extent to which RDD&E activities need to be adapted to meet 
the demands of the particular contexts within which they are applied, 
at least if they are to function effectively as problem-solving tools 
within those contexts, is also only now coming to be realized 
(Gideonse, 1972; Schalock and Sell, 1972). RDD&E within the context 
of agriculture are simply not the same set of activities that they are 
within the context of medicine or the military. Nor are they the same 
set of activities that they must be within the context of education. 
Units of analysis differ; sources of variation differ; freedom for 
control and manipulation differ; the properties of measurement differ; 
analysis and opportunities for replication differ. While it is prob- 
able that RDD&E activities share a good deal in common across all con- 
texts, it is also probable that the demands of specific contexts force 
upon them a great deal of variability. Thus far, such "education spe- 
cific" adaptations of RDD&E activities have only begun to be made. 

The Studies reported in the present volume are based upon the 
belief that the activities labeled research, development, diffusion 
and evaluation have as much to contribute to the development and well 
being of education as they have had to the development of medicine, 
agriculture, and the like, but for their promise to be realized they 
must be better understood, more fully developed, and better adapted to 
the particular reeds of education than they are at present. In this 
regard a host of fundamental issues need to be resolved: How are re- 

search, development, evaluation, and diffusion to be conceptualized 
so as to make them optimally functional within education as a coordi- 
nated set of problem-solving tools? What are the best contexts or 
organizational structures or institutional arrangements within which 
to apply these tools? What problems are most likely to yield to solu- 
tion through their application? What are the personnel and other re- 
sources needed to carry through on their application? Once such basic 
issues have been resolved it seems reasonable to assume that RDD&E will 
be able to contribute to the solution of educational problems as 
significantly as they have to the solution of other long standing 
problems . 

Such is the hope for educational RDD&E. The distance between hope 
and achievement , howeVer, is great, and given the relatively primitive 
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state of the adaptation of KDD&K activities to the needs of education, 
that distance is not likely to be travelled quickly. The Oregon Stud- 
ies in educational RDD&E were designed to both shorten and facilitate 
the journey that retrains. 



The Context Within Which the Oregon Studies Were Undertaken 



In 1960, with the passage of the National Defense Education Act, 
large-scale funds became available for the first time in the United 
States for the support of educational research and development activi- 
ties. Through these funds the National Science Foundation sponsored 
the curriculum development projects of the late 1950 f s and early 
1960's, and the Bureau of Research, U. S. Office of Education, launched 
a many-sided research program. In 1964, with the passage of the Ele- 
mentary and Secondary Education Act, educational research and develop- 
ment activities received additional support, and large-scale monies 
became available for the first time for the support of education eval- 
uation and diffusion activities. With the passage of the Elementary/ 
Secondary Act, funds administered by the U. S. Office of Education in 
support of RDD&E activities jumped from approximately 10 million dol- 
lars per year between 1960 and 1965 to approximately 100 million dol- 
lars per year since 1965, and the networks of Regional Education Lab- 
oratories, Research and Development Centers, and ERIC Clearinghouses 
were established. 

A parallel but relatively unnoticed growth of educational RDD&E 
activities has taken place within this same period at the local and 
state levels, and within the private sector. While information is not 
available as to the dollar allocations that have accompanied such 
development, it is clear from the appearance of research, development, 
evaluation, and diffusion "divisions" within public school systems, 
state departments of education, and the education industry that the 
magnitude of growth at this level approaches or perhaps even sur- 
passes that fostered at the federal level. 

Thus, in the course of approximately ten years (I960 to 1970), 
educational RDD&E developed from an essentially unknown entity to what 
was rapidly becoming recognized as a major component in the overall 
education system. As with any rapidly growing organism or institution, 
however, the course of its development had not been smooth and a host 
of problems remained to be solved. Critics were able to point to mis- 
takes, poor starts, the frequent shifting of positions, and the down- 
right confusion that often characterized its early years. Fortunately, 
supporters were able to point to some successes, and by the end of the 
first decade of its existence educational RDD&E had emerged in a reason- 
ably healthy condition (for an excellent account of the growth of educa- 
tional RDD&E activities within the United States, see Gideonse, 1969). ^ 



^ I » lu Interest Ing to speculate as to h</w mnh growfli wm; able 
to roriie about at a time when educational KDD&K wan (font V next page) 



in the vernacular of the day, it was not possible for the personnel 
of educational RDD&E to get it all together: the field was still unable 

to produce either the kind or the quantity of products expected from it. 
As a consequence the sceptics and critics, and even the long time sup- 
porters of educational RDD&E, were growing ever more disenchanted. 

They wanted more than promises, rosy pictures of the future, models, and 
pilot demonstrations. They wanted concrete products that made a demon- 
strable difference in the lives of children and youth. Like all healthy 
adolescents, educational RDD&E at the turn of the decade was in a posi- 
tion of having to demonstrate its mettle or take its lumps. It was in a 
position of having soon to take its place as a fully functioning member 
of a society, and contribute accordingly, or be shunted aside as another 
misfit. 

Such was the general context within which the Oregon Studies rested. 
The specific context was a multi-sided program that was designed to 
effect change in the preparation of personnel to function within the 
educational RDD&E community. , The program for change was enunciated by 
the U. S. Office of Education in the spring of 1970 in response to two 
aspects of the general circumstance surrounding educational RDD&E: (a) 

the dramatic emergence of development, diffusion, and evaluation activ- 
ities, as accompaniments to research, as vehicles for educational im- 
provement; and (b) the lack of a sensitive response on the part of 
most training programs to the personnel needs that accompanied the 
emergence of these activities. The latter was interpreted as being the 
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(Cont'd) able to contribute so little to the on-line practice 
of education. While a systematic analysis of the factors accounting 
for its growth has not been made, it seems reasonable to assume the 
interaction of four relatively persuasive conditions: (a) the percep- 

tion that the problems facing American education were critical; (b) the 
relatively strong faith, based on success in other of man's endeavors, 
that the tools of RDD&E would be able to be applied successfully to the 
solution of a great many of those problems; (c) the promising nature of 
some of the products that were emerging from the application of these 
tools, for example, individualized and performance-based curricula, 
computer-based scheduling systems, management information systems; and 
(^) the recognition that a reasonable period of time must pass and a 
reasonable investment of resources must be made in order to develop 
the support systems needed for the implementation of large-scale RDD&E 
efforts. Such support systems include institutions designed to exe- 
cute RDD&E functions, a bureaucracy that can manage and administer 
such functions, and a cadre of reasonably seasoned personnel to execute 
them. In a sense, the supporters of educational RDD&E found themselves 
in the same circumstance that investors in any new product or institu- 
tion often find themselves: the need for and promise of a new product 

are so great that an initial investment in its future seems critical, 
but once investment begins it has to be maintained until the total in- 
vestment starts to pay off — simply to protect the initial investment. 
The rationale and political consequences of doing less in the case of 
educational RDD&E made other than continued support essentially impos- 
sible . 
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ease when 3 alter a l'. S. office of Education investment: oi approxi- 
maleJy K) ini. 1 Lion dollars, educational RDD&E training programs in the 
spring of 19 70 were found to be preparing nation-wide roughly the same 
number and kind of personnel as they were in 1965. With the personnel 
needs that existed at that time, the projected needs of lIu* future, and 
the apparent resistance on the part of most training piograms to change, 
the situation was ripe for overhaul. 

In response to these circumstances Dr. Raul Messier, then Director 
of the Division of Higher Education within the National Center for Edu- 
cational Research and Development (NCERD) , U. S. Of lice oi Education, 
and ns such responsible for the programs of the Training Branch of the 
U. S. Office of Education, launched a three-dimensional plan for change. 
The first dimension involved the creation of a conceptual and empirical 
base on wh i.ch to build new training programs. The rationale for estab- 
lishing such a base was s t rai ght forward : (a) to be effective training 

piograms have to have a clear notion of what it is they are to prepare 
people to do, and (_b) a' that point in time a clear notion of wl at 
people actually do in the conduct of development, diffusion, and evalu- 
ation activities, or the knowledges, sensitivities, and skills needed 
to do them, did not exist. .... 

The second dimension of the plan involved the creation and imple- 
mentation of totally new approaches to training. Again, the rationale 
for such a move was straightforward; (a) there were few if any programs 
in existence that were designed explicitly to prepare persons to func- 
tion within the emerging fields of educational development, evaluation, 
and diffusion; (b^) the design of such training programs was seen as a 
major undertaking; and (c) t he training models used to prepare educa- 
tional R & I) personnel in the past were open to impro vemen t . 

The third dimension of the plan called for the creation of in- 
structional materials tlu«t would reflect the desired changes in train- 
ing, both in terms of procedure and content. Since the exact nature 
of these materials had to await the completion of the other two dimen- 
sions of the plan, the materials development effort was treated essen- 
tially as a pilot or experimental venture during its first year of 
operation . 

Broadly conceived, then, the strategy of the U. S. Office of 
Education’s plan for change was to take steps simultaneously to develop 
substantive input for new kinds of training programs, establish new 
designs for training programs, and create instructional materials or 
‘‘systems” that would effectively deliver the. substantive information 
that the new programs were designed to handle. The plan as a whole 
was coordinated so that ail three lines of activity would interact dur- 
ing the course of their development, and would come together in com- 

pleted fashior at approximately the same point in time. 

The Oregon Studies were designed to fulfill the intent of the 
first dimension of the plan. As such it was to provide (a) an "empiri- 
cal map" of the domain of educational RDD&E, as currently practiced in 
projects regarded to be of high qual^ y and reflective of directions 

which eductitional RDD&E are likely take in the future; (_b) a 
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' conceptual framework that would provide both guidance to that which was 
to be mapped, and a language tor describing it; and (c) a general L zab le 
"case study" methodology for use In the Oregon Studies and subsequent 
mapping efforts. The initial map was to be based upon the detailed 
analysis o r 20 projects that crosscut the domain of educational RDD&E, 
with the analysis of each to include 

. the .identification of the critical outputs of work effort. 

. a description of the characteristics cf each output. 

. a description of the interdependencies of outputs. 

. the standards applied by project personnel to each output. 

. the production operations performed by project personnel 
to generate each output. 

. ti'.e policy and management operations associated with the 
generation of each output. 

. the enabling knowledges, skills, and sensitivities 
reported by project personnel as essential to the 
generation of each output. 

. types of personnel associated with the generation of 
each output, their background of training, and other 
characteris tics . 

. a description of the characteristics, contexts, and 
dynamics of each project as a whole. 

In addition, the Oregon Studies were to prepare a compendium of the 
existing literature dealing with the nature of and interrelationships 
between educational RDD&E, both as a foundation on which to develop 
the other aspects of the project and as a document of general worth 
and interest in its own right. 

The second dimension of the plan involved contracting for the 
competitive design of alternative experimental training programs. 

Twelve grants were let for this purpose in June of 1970, and the de- 
signs were completed and submitted for review by Decemoer of the same 
year. Of the designs submitted, three were selected for implementa- 
tion on an experimental basis: one at the Far West Laboratory for 

Educational Research and Development under the direction of Dr. Paul 
D. Hood; one at the Ohio State University under the direction of 
Dr. Daniel Stufflebeam; and one at the Pittsburg Center for Learning 
Research and Development under the direction of Dr. Glen Heathers. 

The third dimension of the plan involved the letting of a number 
of contracts for the development of specific instructional materials, 
but as indicated previously this activity was largely exploratory in 
nature during its first year of operation. With the completion of the 
other two dimensions of the plan it was anticipated that the develop- 
ment of instructional materials would proceed on a much larger scale. 
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Two other projects need to be mentioned as part o f the overall 
context within which the Oregon Studies rested: (ai) the first AERA- 
sponsored study of educational R & D manpower, completed shortly after 
the Oregon Studies were undertaken (Worthen & Byers, 1970); and (b) 
the second rJRA-sponsored study, which paralleled In time and function 
critical aspects of the Oregon Studies (Worthen, et al., 1971). Rea- 
sonably close contact was maintained with the staff of the second 
AER/ study throughout the conduct of the Oregon Studies. 



The Initial Conception of the Studies 



As funded the Studies had six stated objectives: 

1. Develop one or more conceptual frameworks which define, 
differentiate, and relate the processes of educational 
research, development, diffusion and evaluation. 

2. Provide a description of the competencies needed by staff 
to carry out the operations involved in a selected sample 
of representative RDD&E projects, classes of indicators 
acceptable as evidence of such competencies, and the 
knowledges, skills and sensitivities prere^j.site to 
their demons trat ion , 

3. Provide information on staffing patterns currently in use 
in the selected sample of educational RDD&E projects 
studied in (2) . 

4. Provide recommendations as to ideal staffing patterns 
for the projects studied. 

5. Provide recommendations as to priorities for competency 
development within existing and projected staffing patterns; 
and 

6. Prepare a compendium of currently available papers, both 
within and without education, that describe a) the 
processes involved in educational RDD&E activities, 

and b) the conceptual frameworks that have been used to 
differentiate and/or relate them. 

Operationally, these were translated into four products that were 
rxpected to emerge from the project: 

1. A compendium of available documents that a) conceptualize 
the nature of and the relationships between educational 
RDD&E activities, and b) describe the processes involved 
in carrying out those activities. 

2. At least three carefully prepared and critiqued frameworks 
which synthesize the existing literature relative to edu- 
cational RDD&E activities, and two written critiques of 
each framework. 

f.. 
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3. A refined, fit* la teheed case study methodology that could 
be used in the analysis uf educational RDD&E activities 
within ongoing projects. 

A. A first, break-out of tiu: competencies needed Lu carry 

out RDD&K activities, the classes of indicators acceptable 
as evidence o)' their possession, staffing patterns being 
used to currv out RDB&E c>cu vi Liu;, priorities that should 
be given competency dove i opme.nt within alternative staffing 
pat te rns , e L r , 

So conceived the "ivgon Studies were seen as serving three distinct 
though not independent purposes: the purpose of science, the purpose 

of training and the purpose of policy making. In their function as a 
scientific effort the Studies were to provide an initial mapping of the 
field of educational research, development, diffusion, and evaluation 
(much as any basic scientific effort attempts to establish an initial 
map of the domain with which it is to deal), a methodology by which to 
carry out such mapping., and a conceptual framework that, would give 
meaning to that which was mapped by placing it in an interpretive con- 
text. in their contribution to training the Studies were expected to 
provide conceptual clarity and data chat would be of value to the de- 
sign and operation of personnel development programs that support edu- 
cational RDD&E . The contribution of the Studies to policy setting was 
never particularly clear, but there was an explicit assumption that 
the data that emerged frorn the Studies would be of value to policy 
makers in reaching decisions about the kinds of educational RDD&E 
activities to be funded and the support of training programs pertain- 
ing thereto. 

P rocedurally , the Oregon Studies were divided into two phases. 

PHASE I (May 15, 1970 to October 30, 1970) called for the completion 
of the compendium of available literature, second draft copies of the 
conceptual framework papers and an advanced version of the case study 
methodology. The major activiti.es pursued within Phase I, and their 
relationship to one another over time, are summarized schematically 
in Figure 1. 

Building upon the work of Phase I the empirical thrust of the 
Studies was to occur in PHASE II (November 1, 1970 to August 30, 1971). 
During that period an intensive study of 20 or so exemplary projects, 
varying in focus, the setting in which they were being conducted, and 
size, was to be carried out. The conceptual framework papers were also 
to be completed and critiqued. These activities and their interrela- 
tionships are summarized schematically in Figure 2. 



The Evolution of the Studies 



During the course of the Oregon Studies four major refinements 
occurred that significantly influenced the nature of the products that 
emerged from them: ( a ) the evolution of the basic science function of 

the Studies as theii; primary focus; Cb) an accompanying emphasis upon 
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Figure 1.1. The major activities carried out In Phase \ of the Oregon Studies. 
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Figure 1.2. The major activities carried out in Phase II of the Oregon Studies. 
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the development of an exhaustive and generalizable "case study" method- 
ology; (c) the decision to use the outputs of work effort instead of 
job roles as a basis for analyzing work and its prerequisites within 
projects; and (di) a shift from the notion of a preferred conceptual 
framework emerging from a synthesis of the four conceptual papers pre- 
pared for the Studies to the notion of four independent papers, with 
the paper prepared by the staff of the Studies reflecting the empirical- 
conceptual thrust of the Studies. Because of their centrality, each of 
these refinements is described briefly in the paragrapns that follow. 



Sharpening the Focus of the Studies 

As indicated previously the Oregon Studies were viewed at their 
outset as serving three functions, that of science (the extension of 
knowledge), that of training, and that of policy making. In the early 
stages of the Studies an effort was made to attend equally to all three 
functions when making decisions that affected literature review, con- 
ceptual development and/or data collection methodology. It soon became 
apparent, however, that the complexity that this introduced to the 
decision-making process was so great that it could not be handled with 
the time and resources available. As a consequence, after three or so 
months, the advisory body to the Studies recommended that the first 
responsibility of the Studies be to science, and that the integrity of 
the conceptual framework guiding the Studies, the methodology being 
developed, and the data to be collected not be threatened by training 
or policy considerations. This did not mean that the Studies abdicated 
their responsibility to training and policy, or that the discussion of 
such matters ceased, but it did mean from that point on that training 
and policy considerations did not dominate design decisions. 



O 
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The Development of a Case Study Methodo logy 



From its earliest conception the Oregon Studies were seen as pro- 
viding evidence as to 

"the functions, tasks and operations involved in carrying 
out educational R, D, D & E activities, the interrelation- 
ships that exist between such activities, staffing patterns 
used in their pursuit, indicators acceptable as evidence 
of ability to carry out given operations within given 
staffing patterns , , judgment as to the knowledges, skills 
and sensitivities needed to perform the various opera- 
tions within a given staffing pattern, etc." (from page 10 
of the proposal for the Studies). 



In its early drafts the proposal called for a job or task analysis 
methodology to get at such data, with projects like the Sesame Street 
television series, the Philadelphia Advancement School project, the IPI 
Mathematics Program developed by Research for Better Schools and the 
Harvard computer-based information system for vocational decision- 
making proposed as the context within which to apply it. The rationale 
for proposing such a methodology was relatively straightforward: (a) 

it was a well-established methodology, and one that was seemingly appro- 
priate to the focui; of the project; and (b) members of the faculty of 
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Teaching Research were familiar with it. The rationale for proposing 
t: lie documentation of RDD&E activities within large project sites was 
also relatively straightforward: (a) they were sufficiently complex 

that they would provide a variety of RDD&E activities for description 
within each project; (by they represented instances of research, devel- 
opment, evaluation, and diffusion that were prototypic to such activ- 
ities in the future. 

f.n the course of proposal review USOE staff and their consultants 
recommended that the methodology move beyond that which is ordinarily 
encompassed in a job or task analysis and assume the focus of a "case 
study" methodology . ^ In light of the kind of information being sought, 
and the kind of projects within which information was to be sought, 
this seemed to be an appropriate recommendation — even though it was not 
accompanied by a clear-cut specification as to what such a methodology 
might look like operationally. Two additional recommendations were 
made relative to methodological development during the course of the 
Studies : 

1. Throughout the life of the project developmental and 
evaluative efforts needed to be directed to the method- 
ology per se ; and 

2. For purposes of cross-site analyses data that were com- 
parable across sites had to be tracked and treated 
separately and the data peculiar to each site, as a 
consequence of evolution in methodology, had to be 
tracked and treated separately. 



The Development of Output Analyses 

At the time that the case study approach to data collection was 
adopted it was still assumed that the central focus of the data within 
a case analysis would be the jobs or roles performed by project per- 
sonnel. As the project began co interface with real-world RLD&E, how- 
ever, it was soon found that job definitions were relatively unstable. 
Jobs seemed to be defined differently by different organizations, and 
within a single organization differently for different projects. It 
was also common for jobs to change in definition several times within 
the life of a single project. As a consequence, jobs seemed to serve 
poorly as the basic unit for work analysis. 



3 As used in the Oregon Studies a "case study" was to result in 

a "case profile" of an ongoing RDD&E project. Such a profile was to 

provide an exhaustive description of a project and its operations over 

a specified period of time. As such the "case study" methodology 

assumed a form that was somewhat different than the classical use of 
such a methodology. Chapters 4 and 5 of the present volume describe 
the data collected by the methodology, and the way in which it is 
organized in case profile format, respectively. 



* 
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Faced with this fact an alternative to job definition was sought 
as a focus for data collection and analysis. Outputs of projects 
emerged as a viable alternative since they represent relatively stable 
entities within the life of a project, and they probably represent the 
largest single source of variation in project operations. 

Outputs also held promise as relatively stable units for cross- 
project analyses when RDD&E activities were defined in terms of their 
outputs (see the GLOSSARY for illustrative output definitions, and the 
conceptual paper by Schalock and Sell in Volume III of the series 
reporting the Oregon Studies for operational definitions of research, 
development, diffusion and evaluation in output terms). The centrality 
of the output focus to the data collection effort in the Studies is 
evident in the listing of the major categories of data collected in 
Chapter 4, Figure 4.1. 

A Shift fro m a Preferred to a Data Dependent Conceptual Framework 

To insure as strong a conceptual base to the Studies as possible, 
a scries of conceptual papers were prepared for use in the Studies. 
Three of these were commissioned from well-known persons in the field, 
and one was prepared by the staff of the Studies. The aim of each 
paper was to "....define, differentiate, and relate educational RDD&E 
activities, and provide a rationale for the particular point of view 
taken in doing so." As such, each paper was to represent a synthesis 
and/or extension of earlier conceptual efforts within the field. 

Initially it was planned that a "preferred" framework emerge from 
some combination of the four papers, and function as the conceptual 
base to the empirical-methodological thrust of the Studies. Two fac- 
tors made such a plan ;inworkable: the magnitude of the differences in 

points of view reflected in the conceptual papers and the press of 
time on methodological development. Upon review of the first drafts 
of the papers it was clear that ja preferred framework could never be 
fashioned from the four frameworks proposed in time for it to function 
in a useful way in the shaping of methodology. As a consequence the 
authors of the papers were freed of the expectation that a compromise 
between papers be found, and were encouraged to expand the initial 
drafts of their papers as they saw fit in light of the reviews given 
them by independent critiquers. 

In order to meet the responsibility assumed by the Studies for at 
least one of the conceptual papers to be data dependent, that is, to be 
fashioned against and consistent in final form with the framework used 
for purposes of data collection, the paper prepared by the project 
staff was referenced at all points in its development against data 



4 The 
that "what 
be done to 



premise on which this statement rests is the simple notion 
needs to be produced" in large part determines what needs to 
produce it. 




derived through einpJrica.1 observation. The strategy followed in pre- 
paring the paper was on*- or arriving at a particular level of clarifi- 
cation in regard to belli tin* conceptual framework and the methodology, 
applying, the methodology to the analysis of two or three ongoing proj- 
ects, refine both the conceptual framework and the methodology on the 
basis of those data, reapply the methodology, etc. The rationale for 
insisting that the conceptual framework guiding the empirical thrust 
of the Studies be one that was itself tested empirically involved 
three interdependent assumptions: 

1. 'the data that derive from a study are as good only as the 
concepts that mark the boundaries of that which has been 
investigated, 

2. The "goodness" of concepts is a matter chat must be 
demonstrated empirically, that is, a "good" concept 
is one that permits operational definition, functions 
as a significant source of variance, etc.; and 

3. In its present state of development the domain of 
educational RDD&E does not possess a conceptual 
structure that meets such criteria. 

The aim of the conceptual paper prepared by the Oregon Studies staff 
was to provide a first approximation to such a structure. 



As in any major exploratory effort it was necessary in the Oregon 
Studies to meet not only the day-to-day challenges encountered, for 
example, obtaining the cooperation of participating projects and main- 
taining the health, working effectiveness, and morale of staff, but to 
insure that the technical, logistical, and material support needed for 
the overall success of the project were available as well. Toward this 
end a major source of challenge within the project became the clarifi- 
cation of the parameters of the territory to be charted, the selection 
of the variables within it to be attended to in the mapping process, 
the development of the instruments needed to collect trustworthy infor- 
mation, the preparation of a detailed schedule and logistical plan for 
the expedition that blended optimally the objectives of the mission 
with the time and resources available to it, and the preparation of 
personnel to perforin reliably the functions required on the expedition. 

In addition, however, there was also the challenge of striking an 
appropriate balance between the basic science, the. immediate utilitarian 
and the long-term policy functions of the mission — a challenge that 
explorers like Lewis and Clark, Jacques Cousteau, and the men of the 
Apollo space program have also had to face. In the sponsorship of 
almost any exploratory effort these contending forces have to be dealt 
with for they represent the collective interests that make such explor- 
ations possible in the first place. The manner in which the balance 
was struck in the Oregon Studies has already been described; whether 
it will prove to be the most productive balance remains to be seen. 



The Challenge Perceived 




Finally, there was the challenge of fitting the Studies within the 
perspective of the long-term needs of the field of educational RDD&E. 
Sense had to be made of what the field was about, where it was going, 
and what needed to be done to get it there — and then be sure that the 
Studies contributed toward those ends. In this regard three basic 
assumptions were made: 

1. The tools of RDD&E hold considerable promise for the solution 
of problems within education, but before that promise can be 
realized they must, be better understood, more fully developed, 
and better adapted to the particular requirements of education; 

2. The development and/or adaption of RDD&E to meet the require- 
ments of education can be accomplished best through empirical 
investigation; and 

3. Prerequisite to empirical investigation is a conceptual struc- 
ture that marks the boundaries of that which is to investigated 
and identifies the elements within those boundaries that are 

to be subjected to investigation. 

Given such a point of view, the basic challenge of the Oregon Studies 
was seen as being no less than laying the foundation for a discipline 
of educational RDD&E, providing a good first approximation to the con- 
ceptual structure of that discipline, providing a reliable methodology 
for its empirical investigation, and providing a first set of empirical 
observations that tested the fit of the proposed conceptual frame with 
the realities that exist. Just as the journey of Lewis and Clark 
opened the western part of the United States to exploration, levelop- 
ment, and commerce, and the explorations of Cousteau and the Apollo 
program have opened the oceans and the solar system to similar possi- 
bilities, the Oregon Studies were designed to turn the tools of edu- 
cational RDD&E upon themselves and so open the field to exploration 
and development in service of American education. In the judgment of 
those who have been most deeply involved in the project the perception 
of the challenge as anything less would have been to miss the point of 
the whole effort , 




References 



Chase* F. (;. R & D in the remodeling of education. Phi Delta Rap pan , 
Feb. 1970, 220-304. 

Gideonse, H. D. Educational research and development in the United 

States . Washington, D. C. : U. S. Department of Health, Education, 

and Welfare, Office of Education (0E-12049) , 1969. 

Gideonse, H. D. Research and development for education; a market 

model. In The Oregon Studies in Educational RDD&E: Conceptual 

Frameworks* ed. H. Del Schalock and G. Roger Sell, Teaching v 

Research, Monmouth* Oregon, 1972, Vol. Ill, 12-48. 
(OEG-0-70-4917-0701) 

Schalock, H. Del, & Sell, G. Roger. A framework for the analysis and 
empirical investigation of educational RDD&E. In The Oregon 
Studies in Educational RDD&E: Conceptual Frameworks, ed . 

H. Del Schalock and G. Roger Sell* Teaching Research, Monmouth, 
Oregon, 1972, Vol. Ill, 193-261. (0EG-0-70-4917-0701) 

Worthen, B. R.* Anderson, R. D., & Byers, M. L. A study of selected 
factors related to the training of researchers , developers * 
diffusers an d e valuators in education . Washington, D. C. ; The 
American Educational Research Association, 1971. (Final report 
of USOE Grant OEG-0-71-0617-520) 

Worthen, B. R. * & Byers* M. L. An exploratory study of selected 
variab les re lated to the training and careers of educational 
research and research-related personnel . Washington, D. C.: The 

American Educational Research Association, 1970. (Final report 
of USOE Grant OEG 0-9-180240-3757-010) 



Chapter 2 



PREPARATIONS 



As with any exploratory effort a great many preparations had to be 
made for the journey. Personnel had to be selected and trained, infor- 
mation about the territory to be traversed had to be collected and re- 
viewed; charts and maps had to be prepared as guides to the journey; the 
limits of the territory had to be mapped, the dimensions of that terri- 
tory to be attended to in the mapping process had to be established; 
instruments had to be developed that would permit the collection of 
trustworthy information for use in map building; and the specific route 
to be followed during the course of the mapping venture, and the specific 
sites to be studied along that route had to be determined. The purpose 
of the present chapter is to fill in the steps taken in the Oregon Stud- 
ies in making these various preparations . The course charted by the 
Studies, and the specific sites selectee for study along the way, are 
described in Chapter 3. 



Selecting, Organizing and Preparing Personnel 



With one exception key staff positions were filled with personnel 
from the Teaching Research Division of the Oregon State System of Higher 
Education. Dr. Del Schalock served as project director; Dr. Dale Hamreus 
as associate director and coordinator of instrumentation during PHASE I 
of the project; Mrs. Bette Porter as coordinator for the preparation of 
the compendium of available literature; and Mr. Roger Sell as coordinator 
for the preparation of the conceptual papers that served as guides to the 
expedition. During PHASE II of the project Dr. Harry Ammerman replaced 
Dr. Hamreus as associate project director and assumed the role of coor- 
dinator of data preparation and reporting; Dr. Hamreus assumed the role 
of coordinator of site selection, visitation and scheduling; and Mr. 

Gregory Thomas assumed the role of coordinator of data collection. The 
personnel serving under the various task coordinators are identified in 
the Preface to the volume. Overall project coordination was achieved 
through a council composed of task coordinators, the project director 
and the associate director. 

With few exceptions staff assignments on the project were filled by 
persons who had had experience in the performance of similar or related 
tasks. Thus, for example, both Dr. Hamreus and Dr. Ammerman had had ex- 
tensive experience in task and job analysis; Mrs. Porter had had experi- 
ence in library research and bibliographic preparation; Mr. Thomas had 
had experience in the management of large, computer based data files; 
and a central core of the data collection team had had experience in inter- 
view methodology and/or the preparation of case histories. AIL of the 
staff, however had to develop a range of new skills in order f<> rn^’f Ho* 
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rliMii.irulN uf Lite pro )(•( t . Tho conduct of hit i-rvi ovo .irmim! project ihil- 
; hi t •» i -i i 1 u* r lli dii j< •!> do i i n j i i ons 9 the prep-i r .u i < >u « » t i ,t;«« • p i n I i 1 1 -s .is 
d.jta repositories raLtirr than interpretative suriinjr ’i <■:. and the articu- 
lation of a continuous lv evolving methodology and cnu/eptual framework 
were Lusks that demanded the development of new skills on the part of 
everyone involved. 



Collecting and Reviewing Available Literature 



Since the domain of educational RDD&E had been traversed by a re- 
latively large number of ea. lier travelers, and since many of them had 
recorded their observations, experiences and thoughts while there, it 
seemed prudent to take full advantage of the information available in 
recorded form. Accordingly one aspect of the project called for the 
careful review of the literature that pertained to the nature of educa- 
tional RDD&E, and the compilations of the best of that which existed in 
a volume that could be used by the expedition staf r s well as by others 
interested in the territory. Specifically, the articles collected for 
the volume were to serve three purposes: (a) a basic literature source 

for the preparation of the conceptual papers appearing in Volume III; 

(b) a basic literature source for the development of the methodology 
that was to be used in carrying out the empirical study for which the 
Oregon Studies were responsible (See Volumes LV and V); and (c) a basic 
literature source for students and staff in training programs, and 
personnel in the field. As such it was to represent the first attempt 
to draw together the literature on the nature of and the interdepend- 
encies between research, development, diffusion, and evaluation in the 
field of education. 

Five specifications guided the development of the compendium of 
literature: 

1. The articles selected were to be supportive of the con- 
ceptual and methodological thrusts within the Oregon 
Studies, that is, they were to be informative as to the 
nature of and similarities and differences between 
educational RDD&E activities; 

2. Within the limits of the literature available, the 
collective set of articles was to attend as fully to 
development, diffusion, and evaluation activities as 
it did to research; 

3. The articles were to be representative of the full 
range of views that existed in the published lit- 
erature about such activities; 

4. The articles were to be of a quality that would permit 
them to be judged "seminal" in nature; and 
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i!\c articles wore to be classified according to their 
R,D,D, or E focus, or some combination thereof, and 
organized within the compendium so that readers could 
gain easy access to them under an K,D,D, or E Label. 

To meet these spec I fi eat Lons a five-step process was followed in article 
selection. The first step involved an extensive review of the literature. 
At this level the screening process was gross, for essentially any article 
that dealt conceptually with any aspect of the domain was catalogued for 
submission to the second-level screen. The literature searched Included 
all issues of Research in Education , Reports of Research in Education , 
Current Index to Journals in Education , Education Index , RAND Publications , 
Dissertation Abstracts , Applied Science and Technology , and U.S. Government 
R&D Reports published between 1960 and 1970. Staff members and consult- 
ants served as additional sources for the identification of articles. At 
the completion of this step In the selection process over 500 articles 
had been identified for further consideration. 

The second step in the process involved the careful reading of each 
article identified in the initial search, judgment as to its appropriate- 
ness to the volume, and judgment as to its overlap with other articles. 

This step was carried out by three staff members who were not involved 
in the conceptual or methodological thrusts of the Studies. At the com- 
pletion of the second-level screen 240 articles remained. 

The third step of the selection process involved e review of the 
surviving articles by staff members working on the conceptual and method- 
ological aspects of the project. Each of the 240 articles was forwarded 
to three staff members who read and responded to them on a standard form. 
The form called for judgments as to the intent of the article, its sub- 
stantive content, the working groups within the Studies it would best 
serve, the utility of the article to that group, and whether the article 
should or should not be considered further for inclusion in the compen- 
dium. In most cases an article was retained when two of the three staff 
members who read it recommended it for inclusion. One hundred and thirty- 
one articles survived this step in the screening process. 

The fourth step in the process involved a formal review of the 131 
articles by a panel u f judges . The panel was composed of compendiur 
staff members, Dr. Gordon Hoke of CIRCE, University of Illinois at 
Champaign-Urbana, and Dr. Don Coombs, Associate Director of ERIC at 
Stanford University. Final judgments as to the inclusion-exclusion of 
articles were made by the panel at a four-day retreat. All inclusion 
judgments required the consensus of the panel. Eighty-nine articles 
met this criterion. 

The fifth sti 3 p in the selection process involved the review of the 
recommended articles in compendium form by a number of U.S. Office of 
Education staff members, training program directors, students in train- 
ing, and participants in the 1971 meeting of the American Educational 
Research Association. As a result of this level of review, a few of the 
articles included in the initial compilation were excludeo, and a few that 
had either been missed in the initial literature search or (hat had breji 
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excluded for one reason or another along the way, were added. As the 
volume currently stands it contains 86 articles distributed across var- 
ious comb inations of RDD&E activities. Research receives emphasis in 
34 of these articles, development in 30, diffusion in 24, and evalua- 
tion in 35. 

In order to meet space requirements, some extracts from larger 
documents have been included in the compendium. Minor editing also 
has been done on some articles to sharpen their focus on the issues of 
concern in the compendium. All documents so modified have been approved 
for inclusion in the compendium by their senior authors. 



The Preparation of Charts and Maps 
as Guides to the Expedition 



The charge to the Oregon Studies "to map the domain of educational 
RDD&E" carried with it a number of constraints: (a) with the time and 

resources available, the mapping process would have to be limited in its 
coverage of the domain, that is, only a sample of the sites that could 
be studied for mapping purposes would be studied and only a selected 
set of dimensions that could be attended within sites would be at- 
tended to; (b) in a similar vein the number of parameters or features 
within the territory comprising the domain of RDD&E dictated the nec- 
essity of sampling within them; (c) the decisions relative to which 
dimensions of the territory, e.g. topological, economic, cultural, 
should be given attention were confounded by a paucity of indicators 
as to which would prove most profitable; (d) the collection of data, 
and thus the development of data collection methodology had to await 
clarification of such matters; and (e) the project had the responsi- 
bility of clarifying the parameters of the domain and the critical 
variables within before the mapping process was to begin. As a con- 
sequence the project faced an enormous conceptual task, and an equally 
enormous methodological task, before actual mapping activities could 
get under way. 

The vehicle chosen for purposes of conceptual clarification was the 
commissioning of independently prepared conceptual papers by persons 
knowledgeable of the territory to be mapped, the repeated public inter- 
face of the papers, and the insistence that they be reconstructed after 
each public review. The purpose of the present section of the chapter 
is to describe the process by which the collection of conceptual papers 
was developed. The preparation of the conceptual framework that guided 
the empirical thrust of the Oregon Studies is described in the section 
that follows. 

Five specifications guided the development of the conceptual 
papers : 

1. Each was to define, differentiate, and relate educational 
RDD&E activities, and provide a rationale for the particu- 
lar point of view taken in doing so; 



2. In combination, the papers were to provide a set of alter- 
native views as to the nature and function of RDD&E 
activities within the context of education; 

3. In combination, the papers were to be inclusive of the 
various segments of and points of view within the domain 
of educational RDD&E; 

4. In combination, the papers were to provide a synthesis and 
an extension of ealier conceptual efforts; and 

5. At least one of the papers was to be data dependent, that is, 
it was to be fashioned against and consistent in final form 
with the framework that guided the empirical thrust of the 
Oregon Studies. ^ 

To meet these specifications, authors were sought who were both well 
recognized and respected for theii contributions to the field, and who 
held differing points of view as to the nature of and interdependencies 
between the elements within it. 

Dr. Hendrick G. Gideonse, for six years Director of the Division of 
Planning for the U.S. Office of Education’s National Center for Research 
and Development, and, at that time, a professional staff member of the 
Subcommittee on Executive Reorganization and Government Research, United 
States Senate, and Adjunct Professor of Policy Science at State Univer- 
sity of New York at Buffalo, prepared the paper that appears first in 
the volume. His long experience in the management of RDD&E activities 
at the federal level, his interest and participation in policy decisions 
affecting such activities, and his work as a historian of the field, 
qualifies him uniquely to address the topic from the point of view of an 
’’informed bureaucrat.” From that frame of reference Dr. Gideonse is of 



L The initial plan for the conceptual papers called for their 
synthesis into what was termed a "preferred” framework. That framework 
was then to serve as the conceptual base to the empirical and methodo- 
logical efforts within the Oregon Studies. The assumption underlying 
the. plan was not only that a synthesis could be achieved, but that it 
should be — both for the benefit of the Oregon Studies and the field as 
a whole. When the first drafts of the papers were reviewed, however, 
it was apparent that the notion of synthesis would have to be abandoned. 
The views presented in the papers were so diverse, and were held with 
such conviction, that the probability of reaching consensus on a single 
framework — especially in time for it to be of value to other aspects of 
the Studies — was near zero. As a consequence, it was decided that only 
the paper to be prepared by the Oregon Studies staff was to be data 
dependent, and the writers of the other papers were to be free to de- 
velop their respective ideas without regard to the criterion of data 
dependency or the desire for synthesis. 



the opinion that the conception of research and development traditionally 
held within education is counterproductive, and he makes that opinion 
known in Iris paper, "Research and Development for Education: A Market 

Model." Reflecting as it does a questioning of the basic assumptions 
that have governed the. practice of educational RDD&E, his paper pro- 
vides a good point of departure for thinking generally about the role of 
RDD&E activities within the context of education. 

The second and third papers in the volume deal more directly with 
the practice of and interrelationships between educational RDD&E. The 
second paper, "Educational Inquiry and the Practice of Education," au- 
thored by Drs . Gene V Glass and Blaine R. Worthen of the Laboratory for 
Educational Research, University of Colorado, focuses heavily upon the 
nature of educational research and evaluation, and their relationships 
to the improvement of education. Both Dr. Glass and Dr. Worthen are 
well known for their concern about these matters, and for their active 
role in the work of the American Educational Research Association. It 
is not accidental, therefore, that their paper reflects the "classic" 
or "AERA" position with respect to the nature and function of RDD&E 
activities within the context of education. 

Dr. Leslie J. Briggs, Professor of Education at Florida State Uni- 
versity, long recognized for his pioneering work in educational sys- 
tems development, is author of the third paper, "Development and Diffu- 
sion as Mechanism for Educational Improvement." In his paper, Professor 
Briggs addresses the dual problems of developing demonstrably effective 
educational practices, and getting those practices into wide use in the 
schools. After reviewing the literature that pertains to these issues, 
he outlines a research-based model of development that he believes in- 
corporates within it the elements needed to assure both sound development 
and widespread diffusion. While there is some overlap of effort between 
the Briggs and Glass-Wor then papers, it is not so great as to appear 
redundant . ^ 

The paper prepared by Dr. Schalock and Mr. Sell of the Oregon Stud- 
ies staff differs from the other papers in that it sets the stage for 
and provides the concepts used in the empirical thrust of the Oregon 
Studies. As such, it provides basic definitions of the concepts of edu- 
cational RDD&E, differentiates rather carefully between them, and spalls 
out their interdependencies in largely operational terms. It provides 
a review of the existing literature as a basis for the definitions offered, 
and provides a rationale for thinking of RDD&E activities as "problem- 
solving tools" within the context of education. In contrast to the other 
papers, it does not deal with what "ought" to be with respect to the role 
of such activities in education, or make recommendations as to policy 



n 

Only three papers were commissioned initially for the Inclusion 
in the volume. Upon the first review of those papers, however, it was 
the consensus of all writers and reviewers that a paper should be added 
that dealt expressly with the concepts of educational development and 
diffusion. Dr. Briggs, who was serving as a reviewer of the papers at 
the time, accepted the invitation to prepare such a paper. 



with respect 
activities . 
Analysis and 
reflects its 



to the long-term support and/or implementation of such 
The title of the Schalock-Sell paper, n A Framework for the 
Empirical Investigation of Educational RDD&E," accurately 
contents . 



In addition to the careful selection of authors as a means of 
assuring quality in the volume as a whole, and differences represented 
in point of view, each paper was submitted to a formal critique by an 
independent reviewer. Also, the papers as a set were submitted to 
formal critiques by two independent reviewers. Dr. David L. Clark, 

Dean of t.he School of Education, Indiana University, prepared the 
critique of the Gideonse paper; Dr. Paul D. Hood, Director of the Com- 
munications Program, Far West Laboratory for Educational Research and 
Development, prepared the critique of the Glass-Worthen paper; 

Dr. Norman J. Boyan, Dean of the Graduate School of Education, Univer- 
sity of California, Santa Barbara, prepared the critique of the Briggs 
paper; and Dr. Glen Heathers, Professor of Education, Learning Research 
and Development Center, University of Pittsburg, prepared the critique 
of the Schalock-Sell paper. Dr. J. Thomas Hastings, Director of the 
Center for Instructional Research and Curriculum Evaluation, University 
of Illinois, and Dr. Francis S. Chase, Educational Consultant, Profes- 
sor (and Dean) Emeritus, The University of Chicago, prepared, respec- 
tively, the introductory and summary critiques of the papers as a set. 

By providing for such critiques, and allowing an opportunity for au- 
thors to respond to them, it was hoped that the strengths and weakness- 
es of the papers — individually and collectively — would be explicated. 

It was also hoped that it would open a dialogue at the conceptual level 
that would continue, for without such interchange there is little hope 
that educational RDD&E will ever achieve the vitality and sophistication 
that it must if it is to make an observable difference in the practice of 
education . 



Finally, the procedure followed in the preparation of each of the 
papers in the volume, and to some extent their critiques, was also de- 
signed to further their quality and sensitivity to differing points of 
view. The first step in the process was to submit draft copies of the 
three base papers to public review (a two-day conference held on July 
26 and 27, 1970), some three months after the papers had been commis- 
sioned. With one exception all of the critiquers of the papers, the 
four continuing consultants to the Oregon Studies, ^ U. S. Office of 



The continuing consultants to the Studies were Dr. Harry L. 
Ammerman, an industrial psychologist with the Human Resources Research 
Organization; Dr. Samuel Sieber, a social psychologist at the Bureau of 
Applied Social Research, Columbia University; and Dr. James Watson, a 
social anthropologist at the University of Washington. Dr. John E. Hop- 
kins of Indiana University, while technically a consultant to the U. S. 
Office of Education, also functioned as a continuing consultant to the 
project throughout its history. Following the October conference, Dr. 
Ammerman joined the project staff. 




i.ducation representatives,^* and selected staff from the Studies took 



pari in the review. Copies of the papers were made available to partic- 
ipants prior to the conference. The critiquer assigned n particular 
paper led the review of that paper at the conference. 

A second conference was held in mid-October, at which time a second 
draft of the initially commissioned papers, and a first draft of the 
Briggs paper, were reviewed. As in the first conference, the standing 
consultants to the project, U.S. Office of Education representatives, and 
selected staff members from the Oregon Studies took part in the review. 

In place of the critiquers, however, were the directors of 12 USOE funded 
projects to design new training programs for the preparation of RDD&E 
personnel. (The relationship between the training design projects and 
the Oregon Studies was discussed in the preceding chapter.) The 
rationale for the substitution was that input to the papers at that 
point in time by persons who had been thinking about similar issues, 
but within an applied context, would be of greater benefit than a second 
round of review by persons who were to have an opportunity to formally 
critique the papers at a later time. 

The July and October conferences constituted the only formally 
planned sources of input to the preparation of the papers and critiques. 
Authors used other sources, of course, but what those were, and when 
and how they were used, was a matter of their own choosing. No formal 
mechanism was established to provide for face-to-face interaction be- 
tween critiquers and writers, or between critiquers and critiquers, be- 
yond the July conference. Also, no formal provision was made for the 
interaction of the authors of the papers with the critiquers or with 
the editors of the volume beyond the July and October conferences. As 
a consequence, the form and substance of both the papers and critiques 
are very much a matter of the author’s own style, point of view, and 
interpretation of role. 



Specifying the Parameters of the Domain of Educational 
RDD&E, and the Dimensions to be Mapped Within It 

With the demise of the notion of a preferred conceptual framework 
emerging from a combination of the four conceptual papers, which was to 
servf as the conceptual base for the empirical thrust within the Oregon 
Studies, the responsibility for preparing such a framework shifted to 
the staff of the Studies. This meant, in effect, that the conceptual 



^ The U. S. Office of Education representatives attending the two 
conferences were Drs. Cora Beebe, from the Planning and Evaluation 
Division, John Egermeier, head of the Research Training Branch, and 
Paul Messier, at that time Director of the Division of Higher Education 
within the National Center for Research and Development. Dr. Messier 
is now directing implementation planning for the proposed National 
Foundation for Higher Education. 




frameworks prepared by others served essentially as reference documents 
to the framework prepared by the staff of the Oregon Studies, and that 
the authors of other papers served essentially as advisors to it. The 
Schalock-Sell paper, described in the previous section, constitutes the 
conceptual framework that guided the empirical thrust of the project. 

Procedurally , the framework developed by project staff was prepared 
quite differently than the other conceptual papers. While it was sub- 
mitted to the same review and critiquing process as the others, and thus 
referenced against the content of the other conceptual papers and pro- 
fited from the advice of those authors, it was also referenced against 
the operational demands of a data collection methodology and the reali- 
ties of ongoing RDD&E activities. The strategy followed in preparing 
the framework was one of arriving at a given level of conceptual clari- 
fication, translating this into a workable data collection methodology, 
applying the methodology to the observation of ongoing RDD&E activities, 
and, on the basis of the data derived from these observations, refining 
both the conceptual framework and the data collection methodology. This 
cyclical process permitted a continuous test of the adequacy of the 
conceptual frame and the methodology. 

During the course of the project a half dozen revision cycles were 
completed. Each cycle involved the application of both the conceptual 
frame and the methodology to differing kinds of RDD&E activities. By 
the end of the project both the conceptual framework guiding the method- 
ology, and the methodology itself, had been applied to and found to be 
functionally descriptive of research, development, diffusion, and eval- 
uation activities that had a wide variety of foci, that varied greatly 
in size and complexity, and that were carried out within a number of 
differing contexts 

As it stands presently the framework consists of formal definitions, 
relational terms, and a taxonomic structure that provides a language for 
describing both the vehicles or mechanisms that carry RDD&E activities 
(projects, programs, institutions), and the structure, function, and 
operation of those mechanisms, Specifically, the framework consists of 
seven interrelated conceptual sets: one which provides basic defini- 

tions for educational RDD&E; one which provides for the description of 
the interactions between RDD&E activities as they occur within the con- 
text of education; one which provides for the description of vehicles 
which carry educational RDD&E; and four which describe the organization 
and operation of such vehicles — their outputs, the operations required 
to produce their outputs, and the inputs and organizational structures 
required to carry out their operations. Collectively, the first three 
of the seven conceptual sets were designed to map the external parameters 



While the framework will undoubtedly need to be revised or 
extended to accommodate RDD&E activities not encountered during the 
course of its development, or as other dimensions of RDD&E activities 
are selected for study, the framework as it presently exists is believed 
to reflect the critical parameters of the domain of RDD&E, and the crit- 
ical factors affecting operation within it. 
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of the domain of educational RDD&E, and the last four were designed to 
map the critical factors affecting operations within it. In combination 
they were intendeo to identify all of the major dimensions of, elements 
within, and interactions between RDD&E activities as they occur within 
the context of education. The Schalock-Sell paper that explicates the 
framework appears as Chapter 4, in Volume III of the series of volumes 
reporting the Oregon Studies. 

As developed the framework was seen as providing the conceptual 
tools needed to describe or map the domain of educational RDD&E as a 
whole, or any given R, D, D, or E activity within it. It also provided 
a language that permitted reliable communication about that which was to 
be mapped, and a way of thinking about RDD&E generally within the context 
of education. ^ 



Developing the Methodology to be Used in Mapping 



Much has already been said about the development of the data collec- 
tion methodology. The preceding chapter described (1) the reasons for 
employing case study approaches to the problem, (2) the case profile 
format for data preparation, and (3) the rationale for selecting output 
analysis as the primary organizing concept for the evolving methodology. 
The linkage between conceptual effort and methodology development was 
described in the previous section of this chapter. The following 
discussion briefly describes the methodology used and elaborates the 
processes by which it was developed. A detailed description of the 
methodology and its development may be found in Volume V of this series. 

Operationally, the methodology is divided into two phases: a gen- 

eral mapping phase, and a detailed mapping phase. The general mapping 
of an RDD&E activity (PROJECT or PROGRAM) involves five basic steps: ' 

1, Identify the output (PRODUCT, EVENT, CONDITION) expected by 
contractual obligation to emerge from a particular R, D, D, 
or L activity (the FOCAL OUTPUT of a project or program). 



It is probable that the framework and its category sets are 
applicable to the description of RDD&E activities in fields other 
than education, but as yet they have not been tested in that regard. 
Chapter 4 in the present volume contains a description of all the data 
sets within the Oregon Studies that contributed to the language of 
mapping, 

^ For definitions of terms appearing in capital letters in the 
discussion which follows, see the GLOSSARY. 
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2. Identify outputs that constitute an essential part of or are 
prerequisite to the development of a focal output, for 
example, a research design, a testable hypothesis or set of 
hypotheses, reliable and valid measures (the outputs of a 
RESEARCH project), or a product specifications list, in- 
structor and user manuals, and .field test results in support 
of the creation of a reliable instructional system (the 
outputs of a DEVELOPMENT project:). 

3. Organize all of the outputs associated with a particular R, 

D, D, or E activity into a schematic which illustrates their 
hierarchical and/or interdependency relationships (an OUTPUT 
MAP) . 

4. Select the outputs for which detailed information is desired. 

5. Identify the person or persons most knowledgeable of and/or 
involved in the development of selected outputs. 

A detailed mapping of an R, D, D, or E activity is made only in rela- 
tion to outputs selected for intensive analysis . When, applied it in- 
volves three basic steps: 

1. Identify the standards set for a particular output, and the 
indicators acceptable or used as evidence of the realization 
of those standards. 

2. Identify the operations (FUNCTIONS, ACTIVITIES, TASKS, 

ACTIONS) required to produce a particular output according 
to the standards that have been set for it. 

3. Identify the enabling knowledges, skills, and sensitivities 
(ENABLERS) reported by project personnel as being essential 
to the production of a particular output and che performance 
of related tasks. 

Coincidental to the mapping of a project or program is the collec- 
tion of a wide range of supplementary data about it. This includes 
types of personnel associated with the production of particular outputs, 
their background of training, etc.; the perceived criticality of par- 
ticular knowledges, skills, and sensitivities to the production of par- 
ticular outputs; circumstances surrounding the initiation of the proj- 
ect, the rationale for the design employed, and the difference the 
project is expected to make; and a description of the characteristics, 
contexts and dynamics of the project or program as a whole. The deci- 
sion rules governing the application of the methodology, the category 
sets contained within it, and the procedural steps to be followed in 
its use are described in detail in Volume V of the series reporting 
the Oregon Studies. 

As the methodology now stands, data can be generated for virtually 
all of the variables that appear within the conceptual framework. This 
was the intended outcome, of course, and the decision to let the 
methodology remain open to change throughout the life of the project. 
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the strategy of forcing the framework to be translated into a data gen- 
eration me thodology , and the strategy of then checking the adequacy of 
both the framework and the methodology by reflecting them against on- 
going RDD&E activities, was designed to insure a direct relationship 
between the two frameworks. This then represented a conscious attempt 
to ground a data collection methodology in theory. The methodology 
and the data it generated, however, were also referenced repeatedly 
against others who were knowledgeable of the field, or who were in one 
way or another involved in it. Thus, the authors of the conceptual 
papers reviewed tne methodology each time that their papers were re- 
viewed; all of i*he directors of the USOE-sponsored projects responsible 
for the design of new training programs were able to review it at 
least once; representatives from the fields of social anthropology, 
social psychology and industrial psychology reviewed it on three oc- 
casions; Office of Education personnel and their advisory staff were 
party to its development from beginning to end; and project personnel 
discussed it with the parallel AERA project director and staff on 
numerous occasions. In the overall plan of the project the referen- 
cing of the methodology against the conceptual-empirical demands 
placed on it, and against the variety of points of view represented 
by the persons just mentioned, was designed to insure that it would 
be adequate to the demands that would be placed upon it if a discip- 
line of educational RDD&E wp^_ ever to be developed. 



48 






28 



Chapter 3 



THE COURSE CHARTED 



Every expeditionary force must in time decide upon the specific 
route it is to travel and the sites it is to study. The Oregon Studies 
were no exception. Once the territory or domain to be napped had been 
identified, i.e., once operational definitions had been assigned RDD&E, 
and the variables to be attended to in the mapping process had been speci- 
fied, i.e., the conceptual framework guiding the Oregon Studies had taken 
shape, a decision had to be made as to the specific RDD&E activities to 
be studied for purposes of map building. Just as Lewie and Clark could 
not cover all of the rivers, mountains, valleys and plateaus of the 
Northwest in their first mapping expedition, and the Apcllo 14 mission 
could not cover all of the moon’s surface, the Oregon axpedition, given 
the time and resources available to it, could not study empirically all 
ongoing or completed RDD&E activities in education. A problem that faced 
the planners of the Oregon Studies, then, just as it faced the planners of 
the Lewis and Clark and the Apollo expeditions, was one of site or route 
selection. 

Six decisions made early in the Studies had major implications for 
the matter of route and site selection, and thusly, map preparation: 

(a) the decision to study ongoing instead of completed RDD&E activities, 

(b) the decision to study RDD&E activities at the project rather than the 
program or institutional level; (c_) the decision to give primary emphasis 
in the map building effort tc the preparation of detailed site maps (case 
profiles); (d) the decision to prepare site maps for exemplary projects 
only, that is, projects that were judged to be of high quality and illus- 
trative of the directions that RDD&E activities were likely to take within 
the context of education in the future; ( e ) the decision to view the 
preparation of the site maps as a proving ground for the case study method- 
ology that the Oregon Studies was to develop; and (f_) the decision to pre- 
pare outline maps of the domain of educational RDD&E as a whole on the 
basis of the data contained in the site maps. The rationale underlying 
these basic design decisions, and the consequences that followed from them, 
are described in the pages that follow. 



Rationale 



In some cases the rationale underlying the decisions just described 
was relatively complex, and in others relatively simple and straightforward. 
In all cases, however, a rationale existed, and since the various decisions 
were basic to the design, operation and results of the Oregon Studies, the 
rationale underlying them should be made explicit. For the purpose of 
providing comprehensive information each decision is considered separately. 
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The Decision to Study Ongoing 
RDD&E activities 



In broad tenns a decision to use either ongoing or completed educa- 
tional RDD&E activities as a basis for the mapping effort had to be made. 
Three factors led to the decision to study ongoing activities: the 

greater accessibility of persons associated with ongoing activities as 
information sources; the likelihood that persons associated with ongoing 
activities would be able to provide more detailed information about them 
than would persons having to depend upon long-term recall; and the desire 
to study activities that were the closest possible forerunners to educa- 
tional RDD&E activities of the future. 



The Decision to Study RDD&E 
Activities at the Project Level 

Three alternative levels of educational RDD&E activity were considered 
as a basis for map building: projects, programs, and institutions. Argu- 

ments were marshalled for each, but, in the end, projects were selected 
as the most meaningful and manageable unit of study at the present time. 

For purposes of the Oregon Studies a project was defined as M a formally 
recognized, funded, and directed effort aimed at achieving one or more 
specified ends tha~ have their definition in educational research, develop- 
ment, diffusion, and evaluation. " Two arguments were extended that led to 
the choice of project as the unit of study. First, even though educational 
RDD&E programs exist (a coordinated set of projects), and even though 
educational RDD&E institutions exist (an agency responsible for a coordinated 
set of programs), for example educational laboratories and centers, the pro- 
ject still functions as the primary organizational unit within the domain 
of educational RDD&E, i.e., it serves as the primary organizer of work, 
people, resource allocation, products, etc. Second, the project as an 
activity is usually reasonably well defined, time bound, and manageable in 
terms of size and complexity. 



The Decision to Emphasize 
Site Maps 

The rationale for choosing to focus major attention in the Oregon 
Studies on the preparation of ,f site maps” (case profiles) for the projects 
studied rested upon two interrelated propositions: at the time that the 

Oregon Studies were undertaken little was known about educational develop- 
ment, diffusion, and evaluation activities, about how such activities related 
to educational research, or about how any or all such activities related to 
the improvement of education. In order to understand matters not understood 
it was deemed wise to begin by describing them in detail. The rationale for 
reporting such descriptions in case profile format was, simply, that such a 
format invited a more detailed description of project characteristics and 
activities than might otherwise be provided. This was assumed to be true in 
the identification of the variables to be attended to in describing projects, 
and in the exploration of the interactions of those variables. 
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As such, the site maps or case profiles were to serve the purposes 
of both science and training. In support of science the profiles were 
to serve three functions; the careful description of a phenomenon of 
interest; the proving ground for a methodology to carry out such descrip- 
tions; and the development of a data base that would support a preliminary 
"mapping" of the domain of educational RDD&E. In support of training the 
profiles were to serve two functions. At the first level of importance they 
were to provide a means of gaining insight into the nature and function of 
individual educational RDD&E projects. Secondarily they were to provide a 
means of gaining insight into the nature of the domain of educational RDD&E 
as a whole. It was the belief of the designers of the Studies, and the 
advisory body that worked with them, that greater benefit would accrue to 
the field at its particular stage of development from the availability of 
a small number of detailed site maps than a large number of very general 
maps.-** It was the belief of the persons responsible for planning the 
Oregon Studies that the need for detailed, rudimentary kinds of informa- 
tion about the nature of educational RDD&E simply outweighed the need for 
the kinds of information typically made available through survey procedures. 
The 20 case profiles presented in Volume IV of the series of Volumes 
reporting the Oregon Studies reflect that decision. 



The Decision to Prepare Site Maps 
for Exemplary Projects Only 

Once the decision was made to emphasize the preparation of site maps 
within the Studies, an accompanying decision was reached to prepare such 
maps only for exemplary projects. The rationale for this decision was 
straightforward: if only a small number of projects were t^ be studied 

they should be illustrative of the kind of project activities anticipated 
or desired in the future. Some thought was given to the preparation of 
site maps for projects that were experiencing severe difficulties, or that 
were judged likely to fail, but it was concluded that with the resource 
base available a greater information yield would likely come from the study 
of successes rather than failures. The criteria used and procedures 
followed in identifying exemplary projects are described at a later point 
in the chapter. 



The Decision to Use the Site Maps as a 
Proving Ground for the Case Study Methodology 

The decision to describe educational RDD&E projects in case profile 
terms required that a methodology be developed that would generate "case 
study" data. The development of such a methodology became a primary focus 
of the Oregon Studies, and the preparation of profiles (site maps) was, 
to a large extent, a natural culmination of that focus. Two assumptions 



The number of case profiles that could be prepared with the time and 
resource base available to the Studies was estimated at 20. 



accompanied die emphasis on methodological development. First, the 
Oregon Studies represented only the first in a series of empirical studies 
to be undertaken on the nature of educational RDD&E. Second, greater 
benefits would accrue to education over the long term by directing the 
limited resources available to the development of strong methodology than 
to the collection of large amounts of data with a weak methodology. 

As a proving ground for methodology the profiles provided a basis 
for making two kinds of judgments: judgment as to the sophistication 

of the methodology, i.e., the extent to which the methodology generated 
accurate, reliable, and reasonably exhaustive descriptions of educational 
RDD&E activities i and judgment as to the robustness of the methodology, 
i.e., the extent to which the methodology was able to be applied to widely 
varying projects with equally productive results. Evidence as to sophisti- 
cation was obtained by submitting completed profiles of projects to the 
directors of those projects for review and approval (in all cases the pro- 
files met the recessary criteria of sophistication) . Evidence as to robust- 
ness was obtained by applying the methodology to widely varying projects. 

It was assumed that if the methodology was indeed adequate in terms of its 
robustness widelv varying projects could be described with equal facility 
and the data generated in relation to each project would be roughly 
comparable . 



The Decision to Prepare Outline Maps 
of The Domain as a Whole on the Basis 
of the Data Collected for Site Maps 

The decision to view the profiles as a data base for identifying or 
"mapping" the parameters of the domain as a whole emerged as a logical 
extension of the two previously discussed profile functions. Since 
extensive descriptive data were to be made available on the nature of 
educational RDD&L as a result of profile development, and since profiles 
were to be prepared for widely varying projects to test the robustness 
of a methodology, it seemed reasonable to think that the data presented in 
the profiles could serve as a basis for at least launching the process of 
mapping the domain as a whole. Accordingly, the selection of the projects 
to be described was approached from the point of view of their being a 
sample of the projects that exist within the domain of educational RDD&E 
as a whole. Given the small number of projects that could be described 
in case study form with the resource base available, and given the vari- 
ability that was to be reflected in those projects, no illusions were held 
about the representativeness of the sample that could be drawn. At the 
same time, it was reasoned that if the projects to be described sampled 
at all well the variability that existed within the domain, the descrip- 
tions of those projects would provide at least a beginning base for 
sketching an "outline map" of the parameters of the domain. The sources 
of variability sampled by the projects selected are described at a later 
point in the chapter. 

As a data base for mapping the domain of educational RDD&E, the pro- 
files actually served two functions, a basis for mapping the parameters 
of the domain and a basis for mapping the commonalities or central tendercies 



of the domain. As a basis for parameter mapping the profiles constituted 
an excellent source of data* Even though the project sample was small, 
and the absolute data base on which to prepare maps limited, projects 
were selected so as to insure that they were reasonably representative of 
the range of projects to be found within the domain. Vhus, the range of 
personnel employed in the projects described, the range of project 
strategies followed, the range of organizational structures used, the 
range of outputs produced, the range of tasks performed, the range of 
standards held, and the range of knowledges, skills, and sensitivities 
drawn upon in project execution can be assumed to be reasonably represen- 
tative of the range of such things to be found within the domain as a whole. 

Given the project sampling strategy followed, however, it is obvious 
that the profiles constituted a much weaker data base for mapping common- 
alities or central tendencies, Clearly, the sample was drawn to highlight 
the parameters of the domain rather than its central tendencies- Never- 
theless, it was assumed that the data, limited as they were, were amenable 
to central tendency analyses, and such analyses were undertaken. The data 
reported in Chapters 6, 7, and 8 of the present volume represent the 
domain as a whole. 



Site Selection Criteria 



Given the decision to study in detail approximately 20 ongoing educa- 
tional RDD&E projects as the basis for the mapping to be done within the 
Oregon Studies, and the accompanying decision that those 20 projects were 
to be illustrative of the range of projects found within >;he domain, the 
specific criteria to be used in selecting projects became a matter for 
consideration. These were fashioned during the course of two review con- 
ferences that dealt with the matter of methodology, case profile design, 
and project selection, Six selection criteria were finally specified: 

(a) an approximately equal distribution of projects across project foci, 
that is, across research, development, diffusion, and evaluation; (b^, £, d) 
within each project focus the projects selected were to vary widely in size 
(tunding base), institutional setting and stage of completion; ^e) a 
staff of at least two professional persons working at least half of their 
time on the project, or three professional persons working at least a third 
of their time; and (f_) the pool of projects was to be distributed geographi- 
cally. In keeping with the rationale for the intensive study of a small 
number of projects, all projects were to be exemplary in nature and illus- 
trative of the kinds of educational RDD&E activities likely to be pursued 
in the future. 

In reviewing these criteria it is obvious that they were designed to 
optimize the variability within the projects studied. It is also clear 
that from the point of view of hypothesis testing, central tendency identi- 
fication, the fixing of sources of variability, or any of the other pur- 
poses for which classic sampling procedures have evolved, they represent 
an odd set of selection criteria. Twenty cases, selected purposely for 
their variability, have extremely limited utility as a basis for answering 
the kinds of questions typically asked by researchers. From the point of 
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view of hypothesis generation, however, or parameter identification and 
methodological development — which were the prime aims of the Oregon 
Studies — such a sampling strategy had considerable utility. It would have 
been desirable to have included a greater number of projects in the sample, 
and using those projects to be able to sample more systematically the 
sources of variability within the domain, but that was not possible. 

Given the constraints within which the studies were operating, the samp- 
ling strategy adopted probably represents the best possible trade-off 
between the demands of hypothesis generation, methodological development, 
parameter identification, and the desire to obtain information that would 
have utility tc the designers of personnel development programs. ^ Each 
of the project selection criteria is described briefly in the paragraphs 
that follow. 



Project Focus 

In the Oregon Studies the term project focus was used to define the 
primary work emphasis of a project, that is, whether a project was an 
instance of a research, development, diffusion, or evaluation activity. 
Since projects had to oe selected on the basis of their work focus, and 
distinguishing criteria were not readily available between foci at the 
time that the Oregon Studies were undertaken, operational definitions were 
developed. As will be seen in the formal definitions that follow, it was 
the outputs of these activities that were used to sharpen the distinction, 
between them, instead of the nature of the activities involved. 



Research: A problem-solving strategy designed to produce 

reliable KNOWLEDGE, that is, facts, principles., 
theories, and laws that are generalizable and 
that can st n .nd the test of empirical verifica- 
tion . 

Development: A problem-solving strategy designed to produce 

reliable TECHNOLOGY, that is, procedures, mater- 
ials, hardware, and organizational frameworks 
that have a known degree of success in bringing 
about a particular outcome or in performing the 
defined operation. 



Diffusion: A problem-solving strategy designed to bring 

about the IMPLEMENTATION of generalizable know- 
ledge, reliable technology, or trustworthy 
information. (as, used here, diffusion incor- 
porates both the concepts of DISSEMINATION and 
ADOPTION.) 



1 



The sampling issues involved in research aimed at hypothesis generation, 
parameter identification, and methodological development approximate the 
sampling issues involved in research aimed at theory development. For an 
interesting discussion of these issues see Glaser, B. G. and Strauss, A. L., 
The Discovery of Grounded Theory . Chicago: Aldine Publishing Co., 1967 

(especially pp. 45 to 77). 
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Evaluation: A problem-solving strategy designed to produce 

trustworthy INFORMATION regarding a phenomenon 
which occurs in a context or environment over 
which the user expects to exercise influence or 
about whicA he expects to make decisions. 

With these definitions as a guide the expected or contracted for outputs of 
projec ts were looked to for purposes of identifying the focus of projects. 3 



Project Size 

One of the most obvious dimensions on which projects vary is that of 
size. It is evident that as projects increase in size the energy required 
to carry out management functions, differentiate staff roles, and handle 
the complexity of project operations generally tend to increase. As a ^ 
consequence, the factor of size became an important selection variable. 

A number of decisions were reached relative to project selection on 
the basis of size: (a) current fiscal year funding base would be accepted 

as the definition of size; (b) projects having a funding base of less than 
$10,000 for t\ i fiscal year would not be included in the sample; (cj) some 
projects having a funding base in excess of $250,000 during the 1970-71 
fiscal year would be included in the sample; and (d) an emphasis would be 
given to the inclusion of projects that had a funding base midway between 
these two extremes. The final categories of funding used for selection 
purposes were as follows: 

Small projects - Less than $100,000 for fiscal 1971-72; 

Medium sized projects - $100,000 to $225,000 fcr fiscal 1971-72; 

Large projects - Over $225,000 for fiscal 1971-72. 



Once these definitions were applied systematically to projects that were 
labeled, for funding purposes, as research, development, diffusion, or 
evaluation two things were discovered. First, many projects carry labels 
that do not correspond to the output definitions adopted in the Oregon 
Studies, for example, projects labeled evaluation are occasionally, by the 
definitions cited above, development or diffusion projects. Second, most 
projects have more than one work focus. 

^ When site selection criteria were first being considered an effort was 
made to select on the basis of project complexity rather than size alone. 

A three dimensional definition of complexity was developed, involving a 
combination of size, number of agencies participating in a project, and 
the nature of that participation, but the complexity of the criterion, 
and the small number of projects against which to apply it, led to the 
decision to select on the factor of size alone. 
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Setting 



Another dimension on which projects obviously vary is the context in 
which they rest. Some occur within colleges and universities, some within 
R&D laboratories and centers, some within the public schools, etc. While 
there is no evidence that project operations differ significantly as a 
function of the setting within which they are carried out, the fact that 
they are found in different settings was sufficient to force sampling 
decisions to attend to them. Four broad classes of settings were defined 
for sampling purposes: federally supported R&D laboratories and centers; 

privately supported R&D centers; colleges and/or universities; and public 
schools, state departments of education, and professional education asso- 
ciations . 



Stage of Completion 

The last factor attended to through project sampling that was thought 
to affect project operations was that of stage of completion. Concern was 
expressed on the part of the advisory body to the Studies that all phases 
of project operation be studied in detail, and that to avoid the introduc- 
tion of error or incompleteness to the data from dependency upon extended 
recall, projects should be sampled that were in various phases of completion. 
Such a strategy led to what came to be called the "pregnant worm" approach 
to data collection. In this strategy the most extensive data was collected 
on project operations that were ongoing at the point in time that the pro- 
ject was being studied, while less extensive information was obtained 
about project operations prior to the time of the site visit and activi- 
ties projected to occur subsequent to the site visit. For purposes of 
project selection, projects were classified as being in one of three phases: 
an initial phase; a middle phase; or a final phase. Operationally, each 
phase was defined in terms of a third of the time that a project was to 
receive funding. No projects were studied in their prefunding stage or in 
their final report writing stage. 



Staff Size 



Professional staff size entered the picture as a selection criterion 
in a very arbitrary manner, and reflected a concern over the source of 
information about a project residing in only one person. The criterion 
that held that a minimum of two professional staff working an equivalent 
of half time on the project or three professional staff, each working an 
equivalent of a third time on the project, was designed to reduce this 
concern. 



Geographical Distr i bution 

The one remaining criterion used for selection purposes was that of 
the geographical distribution of the projects studied. To avoid the possi- 
bility of regional bias, and to have available as large a pool of projects 
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from which to draw a sample as possible, an effort was made to select 
projects that were broadly distributed across the nation. Three sampling 
regions were used for this purpose, the east coast, the mid-continent, and 
the west coast. 



Exemplary/ Illustrative Status 

The qualitative judgments called for in determining whether projects 
were exemplary or illustrative resulted in several complications. After 
considerable debate about the best way to obtain evidence about these criteria, 
the decision was made to rely upon peer judgment. Two such judgments were 
to be obtained, one from the project officer of the funding agency sup- 
porting the project, and one from colleagues within the field. Four judgment 
criteria were involved in relation to each project: (aj the likelihood that 

the project would produce worthwhile impact upon the educational community; 

(b) the extent to which the project was illustrative of projects to be 
carried out in the future; (c) the history of the performance of the project 
director on projects in the past; and (d) the acceptance generally by peers 
of the project directors* contributions in the past. The manner in which 
these judgments were obtained is described in subsequent sections of the 
chapter . 



Identifying Potential Sites 



At the same time that the selection criteria for projects were being 
defined a pool of projects was being identified from which the sample of 
projects to be studied could be drawn. Three steps were followed in the 
identification process: identifying an initial pool of projects; classi- 

fying and screening at a first level the pool identified; and classifying 
and screening the pool at a second level. Since the procedures involved in 
the various stages of identification varied considerably, each will be 
described separately. 



Identifying the Initial Pool of Projects 

In order to establish an initial pool of projects large enough to 
support the selection of 20 projects that met the various criteria speci- 
fied, a wide range of institutions, agencies, and people were contacted. 
Funding agencies such as the National Institute of Mental Health, the 
National Science Foundation, the National Institute for Child Health and 
Human Development, the Office of Economic Opportunity, the National Center 
for Educational Research and Development, and the Bureau of Educational 
Personnel Development were contacted for lists of projects currently funded 
in the area of educational RDD&E. Foundations such as Russell Sage, Ford, 
Danforth, Kellogg, Carnegie and Rockefeller were contacted for the same 
purpose. So too were a number of colleges and universities, federally and 
privately funded educational R&D centers and laboratories, public schools, 
state departments of education and private agencies, such as the Rand 
Corporation, the Institute for Educational Development, and the General 
Learning Corporation. Persons known to be broadly knowledgeable of RDD&E 
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activities in the nation were also contacted for the purpose of initial 
project identification. The information sought about projects at this 
level of contact included project titles, name of project director, loca- 
tion of the project, level of project funding, and the duration of the 
project. An abstract of the project was also requested If such a docu- 
ment were readily available. Four hundred and twenty-one projects were 
identified as fitting the general perception of those responding to the 
inquiry as falling within the domain of educational RDD&E. 



First Level Classification and Screening 

The initial classification and screening of projects involved two 
steps. The first screen was carried out by staff from the Oregon Studies, 
and involved a perfunctory acceptance-rejection decision as to a project 
falling within or without the domain of educational RDD&E, and the exis- 
tence of sufficient information to permit further inquiry about it. Three 
hundred and seventy-seven projects remained after this initial cut. Their 
distribution by focus and setting appears in Table 3.1. Their distribution 
by size appears in Table 3.2. 



TABLE 3.1 



The Distribution of the Funded Projects Identified 
by Project Focus and Setting 



Project Setting 




Projec t 


Focus 




Totals 




Res • 


Dev. 


Diff . 


Eval. 


Training 




Federally supported R&D 
laboratories and centers 


46 


40 


9 


17 


0 


112 


Privately supported R&D 
centers 


4 


15 


4 


2 


0 


25 


Colleges, Universities, and 
Professional Associations 


128 


53 


7 


25 


2 


215 


Public Schools a.id State 
Departments of Education 


5 


10 


5 


5 


0 


25 


TOTALS 


183 


118 


25 


49 


2 


377 



The second step in the initial screening of projects was to have involved 
staff members of the Oregon Studies in visiting the various agencies that 
funded the projects identified for further information about them. In 
reviewing the agencies supporting the projects identified, however, and in 
anticipating the nature of the time and energy required to gain access to 
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the project files of those agencies, the decision was made to follow this 
procedure only for projects that were funded by the U.S. Office of Education. 
The bulk of the 377 projects identified (323) were funded by the U.S. Office 
of Education and the directors of the various divisions within the Office 
had indicated they would make selected information within project files 
available to staff of the Oregon Studies, so, in a bow to expediency, the 
decision was made to obtain further information only for U.S.O.E. funded 
projects. The information provided by O.E. personnel in this regard was 
limited to information that had been prepared by project personnel, for 
example, project proposals, progress reports, and other documents. 

TABLE 3.2 

The Funding Base of the Projects Identified 



Level of Funding 


Funding for 
Project Duration 


Funding for 
Fiscal 1971-72 


Unknown 


78 


293 


$20,000 or less 


95 


32 


$20,000 - $100,000 


87 


26 


$100,000 - $225,000 


35 


15 


$225,000 - $500,000 


50 


8 


More than $500,000 


32 


3 



With this information in hand the staff of the Oregon Studies met with 
the project officers responsible for the various projects to obtain still 
another level of detail about them. In this meeting the accuracy of the 
information obtained through the project files was confirmed and/or updated, 
an estimate of the current phase of implementation was obtained, a descrip- 
tion of the intra- and interinstitutional linkages within the project was 
obtained, judgments of the exemplary and illustrative characteristics of 
the project were obtained, and a judgment as to the likelihood of the pro- 
ject directors involved being willing to take part in the Oregon Studies was 
obtained. A judgment was also obtained as to the political and emotional 
"health" of the project from the point of view of its being able to with- 
stand the demands that would be placed upon it by becoming a part of the 
Oregon Studies, 

Twenty-seven project officers within the Office of Education were inter- 
viewed in this process. Prior to the interviews contact was made with ..them 
by telephone to confirm their responsibilities as project officers for the 
identified set of projects, to explain the nature of the Oregon Studies, and 
to determine their willingness to provide the kind of information needed. 

If the project officer was amenable, an interview was scheduled at that time. 
All were then sent a letter outlining the intent of the interview, the 
specific items of information desired, and descriptive literature on the 
nature and purpose of the Oregon Studies. 
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Second Level Classification and Screening 



The third step in the screening of projects involved, simultaneously, 
the application of the various selection criteria to the pool of projects 
for which information had been obtained from project officers (123 projects 
remained at this point as viable projects to be studied), and the determi- 
nation of independent judgments as to the quality of those projects. The 
initial filter strategy called for the 123 projects to be sorted first by 
project focus, then by geographical region, then by setting, then by fund- 
ing level, and finally by stage of completion. This process was undertaken, 
but it quickly became apparent that the number of projects identified would 
not support such an elaborate filtering process. As a consequence, the 
decision was made to sort systematically only on the dimensions of focus 
and stage of completion, and then to find the best possible fit across the 
remaining selection variables given the constraints of sample size, schedul- 
ing, etc. Operationally this meant that within the sample of 20 projects 
there would be reasonably equal distribution of research, development, 
diffusion, and evaluation projects, and that within those projects there 
would be reasonably equal distribution of projects in their early, middle 
or late phases, but that would be about as far as balance within the sample 
would go. Within the sample as a whole broad geographical representation 
would be assured, a number of institutional settings would be represented, 
various sized projects would be represented, and all projects would have 
two or more professional staff on a half time basis, but these character- 
istics were not to be distributed systematically by project focus. All 
projects selected were judged as high in quality and illustrative of educa- 
tional RDD&E activities likely to occur in the future. ^ 

Fifty projects emerged from the second level screening process as the 
pool of projects from which the sample of 20 projects was to be drawn. 



An effort was made to obtain an independent rating on the exemplary and 
illustrative features of the 123 projects by submitting them to a panel of 
researchers, developers, evaluators, and diffusers active within the field. 

The strategy called for submitting information about each of the 123 pro- 
jects to the panel and asking them to make individually a qualitative judg- 
ment about the projects they were familiar with and/or about the previous 
work of the directors of the projects. The plan was to continue to cycle 
the projects through the panel in a modified Delphi procedure until some con- 
census was obtained relative to the qualitative features of some subset of 
the projects. After the procedure was tried, it was abandoned as essentially 
unworkable within tne time that was available. This resulted in the quali- 
tative judgments about projects being made initially by O.E. \T 03 ect officers 
and confirmed at an initial level by Oregon Studies staff. Subsequent judg- 
ments about quality were made at the time of the first site visit to a project. 
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Selecting Sites to be Studied 



The 50 projects identified as a result of the series of screens that 
have been described constituted the final pool of projects from which the 
project sample was to be drawn* The final selection process involved three 
additional screens: (a) a conversation with the project director by tele- 

phone about participation in the Oregon Studies; (b) an onsite visit to a 
project for another level of information exchange, providing the telephone 
screen was positive; and (c^) a final inclusion-exclusion decision on the 
basis of data obtained in the onsite visit* It was hoped that of the 50 
projects identified at the start of the final selection process 40 could 
be retained. Given the contingencies of time, scheduling problems, and 
unanticipated factors that invariably decimate sampling plans, it was felt 
that fewer than 40 available projects would threaten the chance of obtain- 
ing completed data for 20.6 



First Level Confirmation: Telephone 

Up to the time of the first telephone call all information on potential 
project sites had been obtained through sources other than project person- 
nel. At this point the directors of potential projects ware called, the 
Oregon Studies were explained to them and they were asked whether they would 
be interested in participating in the studies. If positive indication was 
given, information available about the project was reviewed and confirmed or 
revised as needed. The demands of participation in the Oregon Studies were 
also made explicit, and if interest in participation was still indicated, and 
it appeared that schedules could be meshed, an appointment was made for an 
onsite visitation by a representative from the Oregon Studies staff. 



The process of specifying project selection criteria, and then applying 
them through the steps that have been described, required a considerable 
period of time. During that time the data collection methodology was under 
development, as was the format for the case profiles, a number of projects 
were analyzed in support of these developmental activities, and two of those 
projects were maintained as part of the final project sample. While these 
two projects were not subjected to the formal screens that are described in 
the remainder of the chapter they did meet the requirements of the selection 
criteria as those subsequently emerged. 

7 

During the course of the successive refinements of information on projects 
it was interesting to note that the project officers had much more accurate 
information about projects at their disposal than was in the files of those 
projects, and that project directors, obviously, had much more accurate infor- 
mation about the projects than did the project officers. This seemed to be 
a function of both amount and currency of information. It became clear in 
the course of the Oregon Studies that projects are dynamic, and that their 
status relative to staff, staff assignment, work requirements, implementation 
strategies, and the like, vary almost day-to-day. It was also discovered that 
what appear to be project emphases in proposals do not always turn out to be 
the same emphases when projects are carried out in real life over time. 



Second Level Confirmation: Site Visit 



in most instances two persons from the staff of the Oregon Studies 
were involved in the initial site visit to a project, a senior staff mem- 
ber who functioned in the form of agency-Studies representative, and the 
staff member who was to head the onsite data collection team. In combina- 
tion, these two persons were to complete three tasks during the course of 
the initial site visit: (a) provide the project director with a detailed 

introduction to the Oregon Studies, including the nature of the demands 
that would be placed upon a project director and his staff if he decided 
to participate in the Studies; (b) obtain another level of information 
about the project being considered; and (c) establish a level of familiar- 
ity, and hopefully confidence and trust, between the leader of the projected 
data collection team and the project director. The first task was carried 
primarily by the agency representative, with the aid of several explanatory 
documents. The second task involved an active question-answer session 
between the project director and both staff members of the Studies, as well 
as reference to additional documents or explanatory materials that were not 
in the possession of the Oregon Studies staff. This exchange was aimed 
particularly at clarifying the focal efforts within a project, identifying 
the outputs of those efforts, identifying the products, events, and con- 
ditions that had been created in Support of the production of those out- 
puts, gaining a clear sense of how far along the various outputs identi- 
fied were in their production and determine who on the staff was involved 
in their production. The target of this activity was an information base 
that would permit staff of the Oregon Studies to prepare an "output index" 
and an "output map/ the two documents that were basic to all data collec- 
tion activities within the context of a project.^ 

It was hoped that the third task to be accomplished, that of establish- 
ing a working relationship between the leader of the data collection team 
and the project director, would emerge as a result of the substantive inter- 
action that occurred in relation to the other tasks. 

In most instances final project selection was made at the time of the 
initial site visit. By the close of the visit the representatives from the 
Oregon Studies were usually clear as to whether a project met the criteria 
needed, and the director of a project was usually clear as to whether or not 
he wished or was able to participate in the Studies. If participation was 
confirmed during the site visit a time was agreed upon for the visit of ,the 
data collection team (the time required for data collection varied in length 
from three to five days) . Also, questionnaires that needed to be completed 
by project staff were left with the project director for review prior to 
the return of the data collection team. 



The terms "output index" and "output map," as well as other terms used in 
the data collection process, are defined in Chapter 4. 




42 



Third Level Confirmation: First 

Level Data Workup 



As indicated above most inclusion-exclusion decisions were made at 
the time of the first site visits. In several instances, however, a 

decision to not participate in the Studies was made on the part of a pro- 

ject director after the site visit. In like manner, after an initial 
workup of the data collected at the site visit, the decision was reached 
on several occasions by the staff of the Oregon Studies that resources 
should not be invested in the analysis of a particular project. A decision 
on the part of a project director not to participate tended, on the surface 
at least, to be related to the amount of time required on the part of staff 

away from project activities or in finding a time that the demands of par- 

ticipation could be meshed with the demands of the project. A decision to 
exclude a project on the basis of data collected during the site visit was 
based upon one of two conditions: duplication of project focus with other 

projects already analyzed, or the absence of personnel who could provide 
the information needed about critical aspects of a project. Confirmation 
as a project to be included in the sample came where the initial data work- 
up was complete, all criteria for inclusion in the Studies had been met, 
and a firm time schedule for data collection had been negotiated. 



A Description of the Sites Studied 



The purpose of the paragraphs that follow is to describe the criterial 
attributes of the 20 projects analyzed within the Oregon Studies. Two 
levels of description are provided. First, the sample is described with 
respect to its distribution across various selection criteria independently. 
Following these "single dimension* 1 descriptions the sample is described in 
terras of its criterial attributes as a whole. 



Project Focus 

A major aim of the selection process that has been described was to 
arrive at a sample that included five research projects, five development 
projects, five diffusion projects, and five evaluation projects. This 
distribution was approximated by what was believed to be a five-six-five- 
four distribution at the time that the last few projects were scheduled for 
analysis. Subsequent to those analyses, however, it was found that one of 
the four "evaluation** projects turned out to be more in keeping with the 
operational definition that the Oregon Studies had given to development, and 
so the final distribution of projects by RDD&E was, respectively, five, seven 
five, three. Within the'five research projects 19 contracted-for outputs 
were found, within the seven development projects 39 contracted-for outputs 
were found, within the five diffusion projects 49 contracted-for outputs 
were found, and within the three evaluation projects 27 contracted-for output 
were found. The broad classes of outputs represented by these various focal 
or contracted-for outputs is shown in Table 3.3. Factors that contribute 
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to t he variability obaerved In the data reported In the table are elabor- 
iituil In the paragraphs that follow. “ 



TABLE 3.3 

The Number and Kind of Contracted-For Outputs Identified 
Within the Twenty Projects Studied 



Output Cluster 
Category 


Project Focus 


Totals 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


01 


Reports /Contracts 


13 


6 


11 


11 


41 


02 


Data Instruments/Techniques 


1 


0 


0 


0 


1 


03 


Guides /Manuals 


1 


4 


2 


2 


9 


04 


Work Specifications/ Procedures 


0 


1 


0 


0 


1 


05 


Work Management Systems 


0 


0 


1 


4 


5 


07 


Resource Lists 


0 


12 


1 


0 


13 


08 


Plans /Designs /Theories /Models 


3 


3 


1 


0 


7 


09 


Equipment/Tools/Facilities 


0 


0 


1 


0 


1 


10 


Publications 


1 


2 


16 


2 


21 


11 


Promotional Materials 


0 


1 


0 


0 


1 


12 


Ins true tnl Units /In-School 


0 


6 


0 


0 


6 


13 


Ins true tnl Units /Extra-School 


0 


0 


4 


0 


4 


14 


Promotional Materials 


0 


1 


0 


0 


1 


18 


Presentations 


0 


0 


0 


1 


1 


20 


Recommendations 


0 


0 


2 


0 


2 


25 


Resource Accessibility 


0 


0 


0 


1 


1 


27 


Data (including printouts) 


0 


1 


4 


1 


6 


28 


Taxonomies 


0 


2 


0 


0 


2 


34 


Coordinated Efforts 


0 


0 


1 


0 


1 


38 


Training 


0 


0 


0 


1 


1 


43 


Establishing Opns Parameters 


0 


0 


3 


0 


3 


44 


Provision for Services 


0 


0 


1 


2 


3 


45 


Informed Public 


0 


0 


0 


1 


1 


50 


Disseminating Info/Outputs 


0 


0 


1 


1 


2 


TOTALS 


19 


39 


49 


27 


134 



The outputs identified within the 20 projects studied are described in 
detail in Chapters 7 and 9 of the present volume. 
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Project Size 



In spite of the desire to focus most heavily upon projects of medium 
size ($100,000 to $225,000 per annum expenditure), the final sample in- 
cluded a loading in favor of small projects ($20,000 to $100,000 per annum 
expenditure). In the end nine small projects, six medium projects, and 
five large projects were analyzed. The distribution of projects by size 
and focus is shown in Table 3,4. 

TABLE 3,4 

The Distribution of Project Size By Project Focus 



Project Size 




Project 


Focus 




Totals 


(per annum) 


Res 


1 Dev 


Diff | 


Eval 


Small 

($20,000 - $100,000) 


5 


2 


— 


2 


9 


Medium 

($100,000 - $225,000) 




4 


2 




6 


Large 

(Over $225,000) 




1 


3 


1 


5 


TOTALS 


5 


7 


5 


3 


20 



Project Setting 

The distribution of projects by setting turned out to be more in line 
with the expectations. Six projects were housed within the context of a 
college or university, five within federally supported laboratories or R&D 
centers, five within privately supported laboratories or R&D centers, and 
four within the context of public schools and state departments. The distri- 
bution of project settings by focus is presented in Table 3.5. The distri- 
bution of project settings by size is presented in Table 3,6. 

TABLE 3.5 

The Distribution of Project Setting by Project Focus 





Project Focus 


Totals 


Project Setting 


Res Dev Diff I Eval 


Federally supported R&D 
laboratories and centers 


1112 


5 


Colleges, Universities, and 
Professional Associations 


3 3 — — 


6 


Privately Supported R&D 
Centers 


11 3 — 


5 


Public Schools and State 
Departments of Education 


2 1 1 


4 


TOTALS 


5 7 5 3 


20 




45 



TABLE 3.6 



The Distribution of Project Setting By Project Size 



Project Setting 


Project Size 


Totals 


Small 


Medium 


Large 


Federally supported R&D 
laboratories and centers 


3 


2 


— 


5 


Colleges, Universities, and 
Professional Associations 


5 


1 


— 


6 


Privately Supported R&D 
Centers 


1 


1 


3 


5 


Public Schools and State 
Departments of Education 


— 


2 


2 


4 


TOTALS 


9 


6 


5 


20 



Stage of Completion 

In keeping with the desire to sample projects at various stages of com- 
pletion all three categories of completion, that is, initial, middle, and 
final phases, were represented in the sample. The distribution within those 
phases, however, was intentionally skewed toward the middle phase. Accor- 
dingly, three projects in their initial or early phase of development, eleven 
projects in their middle phase of development, and six projects in their 
final phase were analyzed. The decision to load the sample with projects 
that were in their middle or later stages of development followed the recog- 
nition that a greater amount of information could be obtained from projects 
in those stages of development, assuming continuity of staff, than could be 
obtained from projects in their beginning phases. 



Geographical Distribution 

The regional distribution of projects was also reasonably well balanced. 
Six projects were studied that were on the east coast, six on the west coast, 
and eight in the central region of the United States (includes the Rocky 
Mountains and Appalachia). 



A Summary of the Project Sample 

Table 3.7 summarizes the criterial attributes of the project sample as 
a whole. It also includes a listing of the number of professional staff 
associated with the 20 projects and the number of those staff interviewed. 



TABLE 3.7 



A Summary of the Criterial Attributes of 
The Twenty Projects Studied 



Project Focus 

and Case Profile 
Number® 


Size 


Setting 


Phase 


Pro- 

fessional 

Staff 

Identified 


Pro- 

fessional 

Staff 

Interviewed 


Research 


1 


Small 


Lab/Cntr . 


Middle 


12 


4 


2 


Small 


Coll/Univ. 


Middle 


2 


2 


3 


Small 


Pvt. Firm 


Final 


3 


3 


4 


Smal 1 


Coll/Univ. 


Initial 


7 


7 


5 


Small 


Coll/Univ. 


Middle 


2 


2 


Evaluation 


6 


Large 


Pub Sch. 


Middle 


45 


9 


7 


Small 


Lab/Cntr. 


Final 


4 


4 


8 


Small 


Lab/Cntr. 


Middle 


4 


4 


Development 


9 


Medium 


Coll/Univ. 


Final 


7 


7 


10 


Small 


Coll/Univ. 


Final 


9 


7 


11 


Medium 


Lab/Cntr. 


Initial 


3 


5 


12 


Medium 


Pub Sch. 


Middle 


7 


6 


13 


Small 


Coll/Univ. 


Initial 


5 


6 


14 


Large 


Pvt. Firm 


Final 


12 


12 


15 


Med ium 


Pvt. Sch. 


Final 


2 

. . * 


8 



Diffusion 



16 


Me. ' ium 


Lab/Cntr. 


Middle 


13 


6 


17 


Large 


Pub. S*-h. 


Middle 


36 


12 


18 


Large 


Pvt. Firm 


Middle 


10 


6 


19 


Large 


Pvt. Firm 


Middle 


112 


18 


20 


Medium 


Pvt. Firm 


Middle 


12 


6 






TOTALS 




307 


134 



®The project numbers that appear in the left hand column of the table 
correspond to the numbers carried by the projects as they are presented in 
profile form (see Volume IV of the series of volumes reporting the Oregon 
Studies . 
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Chapter 4 



THE VARIABLES STUDIED 



Coincidental with the choice of projects to be studied, decisions 
had to be made as to what was to be observed on those projects. Just 
as Lewis and Clark and the men of the Apollo 14 mission had to decide 
upon what was to be observed during the course of their journeys, and 
carry with them the instruments needed to make those observations, the 
planners of the Oregon Studies had to decide upon what was to be observed 
within the RDD&E projects chosen for study and prepare the instruments 
needed to obtain the desired observations. Obviously, not all possible 
variables within a project could be studied. Just as Lewis and Clark, 
for example, could not attend in all possible detail to the cultures of 
the Indian tribes or the migratory patterns of the animals or the iden- 
tifying characteristics of the plants encountered along the way, the 
staff of the Oregon Studies could not attend to all the possible detail 
existing within the 20 projects selected for study. Choices had to be 
made, both as to what was to be observed and the level of detail that 
was to be brought to the observations. The purpose of the present 
chapter is to spell out what these choices were, why they were made, 
and how they were implemented in the form of data collection and pre- 
sentation strategies. 



Data Sets 

As indicated in the previous chapter, the charge to ’’map the domain 
of educational RDD&E” carried with it two responsibilities. The first 
was to insure that the maps to be prepared reflected accurately the 
external parameters of the domain, that is, the nature and range of acti- 
vities carried on in the name of educational RDD&E. The second was to 
insure that the maps reflected accurately the internal parameters of 
the domain, that is, the nature and range of activities carried on with- 
in any given educational RDD&E activity. The selection for study of 
projects that varied systematically as to focus, size and setting was 
designed to meet the first responsibility. The decision to prepare case 
profiles, or site maps, for each of the 20 projects studied, end the ac- 
companying decision to utilize a case study methodology in collecting 
the data to be reported in the case profiles, was designed to meet the 
second responsibility. 

Three factors influenced the choice of variables to be observed 
within each project: (a) the charge to prepare a case profile for each 

project studied that would permit a reader of the profiles to understand 
both the "essence 11 of the project being described, and the ’’realities” 
of working within it; (b) the commitment to attend to factors within 
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projects that related directly to personnel and their preparation; and 
(c) the requirement that the primary variables selected for observation 
within projects be ^rounded within a conceptual or theoretical framework. 
As a consequence of these sources of influence, three reasonably distinct 
sets of variables were identified for observation within projects, and 
three reasonably distinct data collection strategies and data sets emerged 
for their description. 



Data Sets that Followed from a Focus 
on General Project Descripto rs 

Five data sets were judged to be essential to understanding a pro- 
ject as a whole. These included (a) the objectives of, rationale for 
ind contributions to be made by a project; (b) the timelines established 
for completing work within a project; (c) the organizational structure 
within which the work of a project was carried out; (d) the political- 
institutional context, as well as the intellectual context, within which 
a project rested; and (e) the "dynamics” of project operation. The first 
three data setb are self-explanatory. Context data pertain to the rela- 
tionship of a project being studied to its sister projects, to the acti- 
vities of the administrative unit within which it rests, and to the poli- 
tical-institutional context of its administrative unit. 

As used in the Oregon Studies, project dynamics referred to the 
nature of project operations, factors influencing project operations, 
and/or the consequences of project operations. It involved information 
pertaining to procedures, feelings, patterns of behavior, or anything 
else that was thought to convey tie "essence" of working within a parti- 
cular project. Operationally, the data pertaining to project dynamics 
involved a description of the pooled perceptions, observations, hunches, 
and insights gained by the staff of the Oregon Studies during the three 
to five day site visit required to collect the data desired on each pro- 
ject. By design these data were collected in a non-systematic way (though 
each staff member on site carried responsibility for collecting such 
data, and the visitation team, as a team, was responsible for preparing 
the narrative statement that constituted these data) , and were intention- 
ally open to subjective judgment. 

No formal category sets were developed by which to code any of these 
classes of data. All are reported in the form of narrative statements 
within the context of the case profiles. 



Data Sets that Followed From 
a Focus on Project Personnel 

Five data sets were also employed in describing the characteristics 
of personnel working within the projects studied. These included (_a) the 
background of training and work experience of professional staff found 
within the projects; (b) a description of the job or jobs held by pro- 
fessional staff within a project; (c) a description of the requirements 
perceived for the job or jobs held; (d) the emphasis given to various 
classes of work activities within the context of particular job roles; 



and (e) the support services and resources available to persons in the 
performance of their respective job roles. All of the cata within these 
sets are reported in the form of questionnaire items. 



Data Sets that Followed From a Focus 
Upon Work Requirements Within Projects 



By and large the data sets that emerged from having to make sense 
of projects as a whole, and having to be clear about the nature and back- 
ground of personnel working within projects, were shaped by the inter- 
action of the staff of the Oregon Studies with the review and advisory 
body to the Studies. This body was made up of professional RDD&E per- 
sonnel concerned with the design of new training programs, consultants 
from the disciplines of locial anthropology, social psychology, and in- 
dustrial psychology, and representatives from the U.S. Office of Educa- 
tion. The data sets that emerged from having to make sense of the work 
requirements within projects were shaped in a much different way, and 
by different forces. These data sets were shaped by the conceptual frame- 
work that guided the empirical thrust of the Oregon Studies, and by the 
interaction of that framework with that which was encountered in the 
course of describing ongoing projects in the field. In this sense 
the data sets describing the work requirements within projects were 
grounded empirically as well as conceptually. This is not to imply 
that these data sets were not influenced by persons in the field. They 
were. The conceptual framework from which they emerged was reviewed on 
numerous occasions by the same review and advisory body that shaped. the 
other data sets used in the Studies. The influence of these groups on 
the shape that these data sets took, however, was much less direct.^ 



Four sets of variables were specified by the conceptual framework 
as critical to understanding the work requirements of projects: (a) the 

outputs of work effort; (b) the standards held for outputs; (c) the opera- 
tions required to produce specified outputs to specified standards; and 
(d_) the knowledges, skills and sensitivities required tc carry out those 
operations. In attempting to describe the outputs of projects, and the 
standards, operations and enablers that related to them, it was necessary 
to establish a number of category sets to handle the complexity that was 
found. Two approaches were taken to the development of these sets, a 
conceptual-empirical (deductive) approach, and an empirical-conceptual 
(inductive) approach. In the former, category sets were developed as 
an extension of the conceptual framework that guided the Studies; in the 
latter, they were developed in response to the data emerging from the 
study of ongoing projects. 2 Operationally, however, the two approaches 



1 For a description of the conceptual framework that guided the empirical 
thrust of the Studies see Schalock, H. D. and Sell, G. R. , "A Framework 
for the Analysis and Empirical Investigation of Educational RDD&E,” in 
Chapter 4 of Volume III of the series of volumes reporting the Studies. 

2 To some extent this is an over simplification, for the conceptually 
derived categories were tested empirically in the course of their derivation 
(see Chapter 2 of the present volume), and the empirically derived cate- 
gories were always influenced by contf^tkial considerations. 



were complementary, for the conceptual-empirical approach yielded cate- 
gory sets that functioned as relatively broad, general organizers of the 
data, and the empirical-conceptual approach yielded category sets that 
functioned at a ‘'close to the source, " descriptive level. 

Conceptually derived category sets . Figure 4.1 provides a summary 
of the data sets used to organize the descriptive information about pro- 
ject outputs, standards, operations and enablers at a broad, conceptual 
level. 



OUTPUTS STANDARDS OPERATIONS ENABLERS 



Products Product 

STRUCTURE Events Process 

Conditions __ 

Policy Setting 
FUNCTION Management 

Production 

Knowledge 

CHARACTER Technology 

Implementation 

Information 

Focal 

LEVEL Component 

Facilicory 



Knowledge 

Skill 

Sensitivity 



Activities 

Tasks* 

Actions 



FIG. 4.1. Category sets used to describe at a broad, conceptual 
level the properties of outputs, standards, operations, and enablers. 

* Of the operations set, only task level descriptions were obtained. 

Time and resources did not permit an analysis of operations at the level 
of actions, and the activities set was left to be derived empirically. 
From this point forward in the text, the term "task" will be substituted 
for the term "operations." 



O 
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Two observations about Figure 4.1 are in order. First, the data 
sets are uneven. Multiple sets exist for the coding of outputs, but 
only single sets— and those not of a comparable nature — exist for the 
coding of standards, operations, and enablers. In part this uneveness 
reflects the special, role played by outputs in describing projects. 

The analysis of project work requirements always started with the iden- 
tification of the outputs toward which work was directed. Since the 
adequacy of this identification determined the adequacy of all that fol- 
lowed, a great deal of care was given the task of output classification. 

As a consequence outputs received more attention conceptually over the 
course of the Studies than did the variables that linked them. A greater 
degree of differentiation was the result. In part, however, the uneveness 
reflected in the category sets appearing in Figure 4.1 is a result of the 
discovery of unsuspected properties of outputs, the inappropriateness of 



c 



some initially proposed sets for operations, and the decision midway in 
the course of the Studies to withhold further taxonomic efforts on stand- 
ards, operations and enablers until those efforts could make use of the 
full data base being established by the Studies. It is probable, there- 
fore, that the uneveness observed in the sets is more likely a reflec- 
tion of an uncompleted conceptual task than it is of the natural order 
of things. 

The second observation appropriate to Figure 4.1 is that the labels 
given the various category sets are arbitrary. They are simply labels 
that have functioned heuris tically in identifying sources of variation 
observed in outputs, standards, tasks, and enablers during the course 
of project description. Whether they are the best labels available for 
the constructs they identify, or whether they even appropriately identify 
the constructs intended, is an open question. They represent a "first 
cut" at identifying and ordering the internal dimensions of RDD&E pro- 
jects, however, and, given the descriptive data on the nature of these 
variables that are now available, provide a good base for subsequent 
conceptual development. 

Arbitrary as they may be, the category sets that appear in Figure 
4.1 constitute the broad organizing categories into which all descriptors 
of project outputs, standards, tasks, and enablers have been coded. The 
meanings they carry, therefore, must be understood if the data reported 
in subsequent chapters are to be understood. Toward that end definitions 
and examples are provided in Table 4.1 for the categories appearing in 
Figure 4.1. 

Empirically derived category sets . While the conceptually derived 
category sets provided the broad, organizing framework for data collec- 
tion and analysis relative to work requirements within projects, the 
empirically derived sets provided the substantive framework. These were 
the sets that took their form and content from the language of persons 
working inside projects. 

Procedurally , the empirically derived category sets emerged in the 
following way. As outputs were identified, and the standards, tasks, 
and enablers pertaining to those outputs explicated, they were assigned 
a classif icatory label. These labels, called primary categories, were 
extremely "close to the source" for they were essentially refinements 
of the words used by interviewees to describe the outputs, standards, 
operations, and enablers embedded in the projects being studied. The 
category sets grew by referencing each interviewee description, as 
it entered the pool of output, standard, task, and enabler descriptions, 
against the set of primary categories that existed at that point in 
time. If the incoming descriptive statement could be classified in 
an existing primary category, it was; if it could not, a new category 
was created to accommodate it. Through this process 326 primary cate- 
gories of outputs, 79 primary categories of standards, 230 primary cate- 
gories of tasks, and 137 primary categories of enablers were identified. 
These sets housed, respectively, 962 descriptors of project outputs, 

1148 descriptors of output standards, 3722 descriptors of output re- 
lated tasks, and 2497 descriptors of output related enablers. Examples 
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TABLE 4.1 



lJet ini tlona 
the 



and Illustrations of the Concaptually 
Charactr r lat 1 ca of Outputs, Jtandardu 



Derived Category Seta that Deacriba 
Operations and Enablora 



Category Set 


Subsets 


Def Inltlona 


Examples 




OUTPUTS 



STRUCTURE 


Produc t 


A tangible or "hard outcome of work effort, concrete 


Task chart; single 






In fora, and transportable at a given point In tima 


concept film 


referlng to Outputs , . . 


Event 


An outcome or work effort that results in the occur- 


Staff meeting; 


the properties of outputs 




re nee of an observable transaction or set of be- 


workshop; speech 


that permit them to be 




hsvlora. 


identified as PRODUCTS, 
EVENTS, OR CONDITIONS 


Condition 


An outcome of ‘ work effort that creates a desired 


Staff welfare; 






circumstance expected to endure within the life 


parent involve- 






of a project, or beyond the project as a result of 
it. 


ment 


FUNCTION 


Policy Setting 


A clarification given an output that establishes 


Policy plans; con- 






tha principles or guidelines to be attended by a 


ceptual leadership 


referlng to Outputs . . . 




project. 




the properties of outputs 


Management 


A claaaif ication given an output that orchestrates 


Interim report; bud- 


that permit their uses to 




the resources (time, personnel, materials, space, 


get; staff meeting 


be Identified as serving 




information) available to a project in the 


the purposes of POLICY 




realization of the outcomes expected from It; also 




SETTING, MANAGEMENT', or 




a report of that orchestration. 




PRODUCTION 


Production 


A class If Ication given an output which builds di- 


Single concept film; 




l 


rectly on the fabrication objectives of a project. 


film typescripts 


CHARACTER 


f- 

Knov ledge 


A classification given an output of RESEARCH, l.e., 


Theory paper, inter- 






an instance of a generallzable idea newly Inferred 


preted data; nature 


referlng to Outputs . . . 




from factor data and which can stand the test of 
empirical verification. 


of reading 


the properties of outputs 
that permit them to be 


Technology 


A classification given an output of DEVELOPMENT, 


Observation techni- 


identified ss instances 




i.a., an instance of a reliable strategy, proce- 


que; typescript 


of KNOWLEDGE, TECHNOLOGY, 




dure, hardwmrd, or set of materials designed to 


IMPLEMENTATION, or 




bring about a particular outcome or perform a 




information 




defined operation. 






Implements t ion 


A claaaif ication given an output of DIFFUSION, l.e., 


Journal article; 






an instance of the ADOPTION and UTILIZATION of 


Sesame Street 






KNOWLEDGE, INFORMATION, and/or TECHNOLOGY. 


puppets; speech 




Information 


A classification given an output of EVALUATION, 


Progress report; 






l.e., an instance of trustworthy facts or data 


appeal data; lit- 






without reference to interpretive or value judg- 
ments relating to that description. 


erature survey 



LEVEL 

refering to Outputs . • . 


Focal 


An outcome of work effort expected by contractual 
obligation to emerge from a project. 


Final report, in- 
formed public 


Component 


An outcome of work effort that constitues an ele- 


Content mop; stu- 


the properties of outputs 
that permit their rela- 


ment of, or an approximation to, a FOCAL OUTPUT. 


dent analysis form 


tionship to project goals 


Facilitating 


An outcome of work effort that makes easier or 


Writing style man- 


to be Identified as FOCAL, 




assists in the generation of FOCAL or COMPONENT 


ual; data analysis; 


COMPONENT, or FACILITATING 




OUTPUTS, but is not in Itself a part ol auch 
outputs . 


staff meeting 



STANDARDS 



STRUCTURE 

refering to Standards . • • 


Output Stan- 
dard 


A criterion applied to, or level of excellence ex- 
pected of, an output; a criterion by which the ade- 
quacy of an output ia Judged. 


Performs consis- 
tently; accep- 
tance by sponsor 


the properties of judg- 


Proceea/oper- 


A criterion applied to, or level of excellence ex- 
pected of, the proceaaee/operat Ions engaged in pro- 


Deadlines are met; 


ments about adequacy that 


ation Stan- 


feedback occurs; 


permit them to be linked 
to OUTPUTS or PROCESSES 


dard 


ducing an output; a criterion by which the adequacy 
of proceiaes/operations are judged. 


project view ac- 
cepted 



OPERATIONS 



LEVEL 


Activities 


Not represented In the data 




refering to Operations • . 


Tasks 


A discrete unit of work performed among other simi- 


Design format for 






larly targeted unite in producing a specific output 


report; compile 


the properties of directed 




to the standards set for it. 


data; determine 


work efforts that permit 






information needed 


them to be identified as 








ACTIVITIES, TASKS, or 


Actions 


Not represented in the data 




ACTIONS 




■ 


' 



ENABLERS 



STRUCTURE 

refering to Enablers . • , 


Knowledge 


A classification given an QtABLER that identifies 
it as an instance of that which la tc ce known as 
a requisite to the accomplishment of an output. 


Knowledge of: sta- 
tistics; data 
needed; learning 
theory 


the properties of pre- 








requisite or enabling 
conditions that permit 
them to be identified as 


Skill 


A classification given an ENABLER that identifies it 
aa an instance of a specific ability corsidered re- 
quisite to the accomplishment of an outpit. 


Skill ins program- 
ming events; hand- 
ling detail 


KNOWLEDGE, SKILL, or 








SENSITIVITY 


Sensitivity 


A classification given an ENABLER that Identifies it 
an an Instance of a specific perceptlvenesa and re- 
sponsiveness considered requisite to accomplishment 
of an output 


Sensitivity to: 
user feedback; 
need for recogni- 
tion; respect for 
staff 



of the*, primary categories used in classifying o tputs, standards, tasks, 
and enablers, ana examples of the descriptive statements that have been 
rodrij within them, appear in Table 4.2. 

Because of the large number of primary categories that emerged in 
the process of classifying outputs and tasks, a higher order set of cate- 
gories had to be created in order to simplify data handling. The creation 
of these higher order sets involved grouping or clustering, and then 
labeling, the primary categories in much the same way that the creation 
of the primary categories involved the grouping and labeling of inter- 
viewee descriptions. During the course of the Studies 51 categories de- 
scribing ,, clusters n of outputs, and 20 categories describing "clusters” 
of tasks emerged. Examples of the cluster categories used in classify- 
ing outputs and tasks, and the primary categories that rest within them, 
are presented in Table 4.3. All cf the primary and cluster category 
sets used for coding purposes within the Oregon Studies, as well as the 
frequency with which each was used within the research, development, dif- 
fusion, and evaluation projects studied, are presented in Appendices 1 
through 9. The descriptive statements contained within the primary cate- 
gory sets appear as appendices within the case profiles reported in Volume 
IV of the series reporting the Oregon Studies. 



Th e Interdependence of Data Sets 

As indicated previously, each case profile that was to emerge from 
the Oregon Studies was to reflect the '’essence" of the project being de- 
scribed, and the "realities” of working within it. Operationally, that 
was translated to mean that each case profile was to describe not only 
the variables listed in the preceding paragraphs, but their interrela- 
tionships as well, It was also translated to mean that a way had to be 
found to collect data on such interdependencies, for unless they were 
sought in the field they could never be reported in the profiles. 

Accordingly, the data collection methodology developed within the 
Studies permitted context, people, and work requirement data to be col- 
lected as interdependent data sets. In keeping with the conceptual 
framework that guided the methodology, outputs of work effort were adopted 
as the primary unit of analysis within projects. Procedurally , once an 
output of work effort had been identified it was possible to link to 
that output the standards set for its production, the operations required 
for its production, the enabling knowledges , skills and sensit iviities 
needed for its production, the persons involved in its production, and 
the relationship of that output and the work associated with it to the 
other outputs involved in the work of a project as a whole. It was also 
possible to link a particular output to the organizational structure of 
a project, the context within which the project rested, and even the 
"dynamics” of a project, though not so directly as in the case of vari- 
ables that depended upon output linkage for their definition. 

The interaction or interdependencies of these various data sets are 
illustrated schematically in Figures 4.2, 4.3, and 4.4. Figure 4.2 is 



The Project as a Whole 
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FIG. 4.2. The classes of information sought in describing 
a project, and their interdependencies. 



of output-work requirement data. The reader is invited to study this 
latter figure carefully, for it represents the basic organizational frame- 
work followed in collecting and reporting the core data within the Oregon 
Studies . 
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intended to illustrate the strength of the interdependencies between data 
sets, i.e., the extent to which they were linked during the course of 
data collection, (the heaviness of line in the figure is intended to in- 
dicate strength of dependency), and Figure 4.3 is intended to illustrate 
the manner in which the nested relationship between context, people, and 
work requirement data is treated within a case profile. Figure 4.4 il- 
lustrates the detailed linkages that exist between the various classes 



Table 4.3. Examples of Cluster Categories of Outputs and Tasks, 
and the Primary Categories Coded Within Thera 



Cluster Categories 



Primary Categories 



OUTPUTS 

Reports /Con tracts 



Work Specifications/ 
Procedures 



Terminal report 
Proposal 

Evaluation report 
Progress report 

Description/ Specifications for 
Output 

Data analysis procedures 
Data Collection procedures 



TASKS 



Clarifying Problem Addressed Specify contexii/scope/limits 

of effort 

Observe relevant/related field 
operations 

Review of/Familiar izat ion with 
materials 

Identify characteristics of 
target population 

Effecting Accountability Establish/Maintain time lines 

Develop record keeping system 

Estimate /Provide material/ 
personnel 

Specify/Establish/Adopt policies 



C o ntext Do t a 
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The nature of the interdependent relationships of the data sets. 



Knowledge Category 
Knowledge Category 
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rlo. -4.4. ihe linkage of data sets used to describe output-work requirement interaction. 



A Numbers Came 



Previous paragraphs have shown that the data collected In the Oregon 
Studies for use in mapping the structure of educational RDD&E projects 
ranges widely in terms of its source, its focus, and its level of detail. 

The implications of this variability for purposes of analysis is discus- 
sed in subsequent chapters. The purpose of the next few paragraphs Jo 
to present in summary form the units of data available from the anal>sis 
of the 20 projects, and the frequency counts that exist for each. 

Tables 4.4 through 4.7 provide such summaries. Table 4.4 contains 
a listing of the variables used in describing general project parameters, 
and the number of observations available in each variable. In all cases 
the N for these variables is equal to the numl "*r of projects analyzed (20). 



TABLE 4.4. Variables Describing General Project Characteristics, 
and the Number of Observations Available on Each 



Class of Data N 



Objectives, rationale, and significance 20 
Time lines and task charts 20 
Project, program and institutional context 20 
Project dynamics 20 



Table 4.5 contains a listing of the variables used in describing 
the characteristics of personnel employed in the 20 projects analyzed, 
and the number o£ observations available on each variable. The N in 
the case of these data varies according to the number of persons com- 
pleting a particular questionnaire. 

TABLE 4.5. Variables Describing General Personnel Characteristics, 
and the Number of Observations Available on Each 



Class of Data ___ N 



Professional Staff Represented 307 
Professional Staff Interviewed 134 
Professional Experience arid Training 115 
Job Descriptions and Prerequisites 115 
Supporting Services and Resources 115 
Work Requiiements in Relation to Job 

Responsibilit ies 90 
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Of the 134 professional staff interviewed, 115 completed the question- 
naire that focused upon personnel characteristics generally, and 90 of 
those 134 completed the questionnaire that focused upon work requirements. 

It will be recalled from earlier discussion that thi variables listed 
in both Tables 4.4 and 4.5 are variables that emerged frcra the demands 
of the case study methodology, rather than from the conceptual framework 
that guided the empirical thrust of the project generally. 

Tables 4.6 and 4.7 contain, respectively, a listing of the empiri- 
cally derived and conceptually derived categories descriptive of work 
requirements within projects that followed from the conceptual framework 
that guided the Studies. Table 4.6 contains essentially a repeat of in- 
formation provided earlier, though in the previous discussion the data 
were not tabled and no mention was made of the number ot outputs analyzed — 
a critical matter since all data reported on standards, tasks and enablers 
are directly linked to those outputs. Table 4.7 simply reports the dis- 
tribution of the descriptor data reported in Table 4.6 when it is ordered 



TABLE 4,6. Empirically Derived Categories Describing Work 
Requirements Within Projects Available, and 
the Number of Observations on Each 





Outputs 

Identified 


Outputs 

Analyzed 


Standards 


Tasks 


Enablers 


Descriptors 


962 


298 


1148 


3722 


2497 


Primary 

Categories 


326 


167 


79 


280 


137 


Cluster 

Categories 


51 


46 




20 





according to the conceptually derived category sets that serve as broad 
organizers for the descriptor data. It is critical to realize that the 
numbers reported for both Primary and Cluster Category sets in Table 
4.6 refer to numbers of categories in a particular category set, not 
frequency counts within categories. 
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TABLE 4.7. 



Conceptually Derived Categories Describing the 
Cnaracterist ics of Outputs and Associated Work 
Requirements Within Projects * and the Number 
of Observations Available on Each* 



Characteristics of 


Ou .puts 


Characteristics of Associated 


Work Requirements 


Outputs (N=962) 


Identi- 

fied 


Anal- 

yzed 


STANDARDS OPERATIONS 


ENABLERS 


STRUCTURE 










Products 


759 


182 


Output 775 


Knowledges 964 


Events 


130 


65 


Process 373 


Skills 848 


Conditic ns 


73 


.51 




Sonsitivi- 685 










ties 


FUNCTION 










Policy Setting 


26 


12 






Management 


514 


174 






Production 


422 


112 






CHARACTER 










Knowledge 


40 


18 






Technology 


576 


178 






Implementation 


136 


55 






Information 


210 


47 






LEVEL 










Focal 


134 


44 






Component 


217 


53 


Tasks 3722 




Facilitory 


611 


201 







. 



* The numbers reported in the table pertain to the numbers of de- 
scriptive statements coded within the various categories listed. The 
distinction between outputs identified and outputs analyzed is critical 
in this regard nr all standards, operations and enablers are linked cO 
the latter set of outputs. 



Data Collection Procedures 

Three relatively distinct procedures were cimplyed in collecting 
data on the general characteristics of projects, personnel working with- 
in projects, and work requirements within projects. All data, however, 
were collected within the context of an "on-site" visit by a data col- 
lection team from the Oregon Studies. Depending upon the size and com- 
plexity of a project, teams varied in size from two to six people, and 
the length of the site visit extended from three to five days. Data 
collection procedures and decision rules are described in detail in Vol- 
ume V of the series of volumes reporting the Oregon Studies. In the 
paragrpahs that follow an overview of the procedures used to collect 
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the various classes of data pursued in the project is provided. 



General Project Descriptors 

Probably the best label for the procedure: used in collecting data 
on general project descriptors is that of "non-obtrusive . " The objectives 
of a project, the rationale for a project, project time lines, organiza- 
tional structures, and the like, were obtained from project proposals 
and other documents descriptive of the project. Also, information on 
the "dynamics" of the projects were gathered through incidental observa- 
tion, the recall of causal comments made by project staff while being 
interviewed, and the "hunches" or "insights" gained while working with 
project data. Almost without exception these sets of data were collected 
without intrusion upon people’s time am' energy. 

The one data set used to describe the general characteristics of 
projects that was intrusive was the data set that described the context 
within which the project rested. Some information of this kind was ob- 
tained from proposals and other documents, but in all cases project 
directors were interviewed when developing a contex: map. In some in- 
stances context mapping amounted to little more than confirmation of in- 
formation gained elsewhere, but in others it involved both the generation 
and piecing together of information about intra- and inter- institutional 
linkages that were simply not made explicit in existing materials. Gen- 
erally speaking, the larger the project the more complex its political- 
institutional-intellectual linkages, and in some cases, for example, the 
Children’s Television Workshop, the development of a map to depict these 
linkages was a major undertaking. 



Personnel Descriptors 

All of the data descriptive of the personnel of educational RDD&E 
were collected through questionnaires administered by members of the 
Oregon Studies staff, either while visiting the project site or through 
telephone conversations. Three questionnaires were used: a) a general 

project questionnaire; b) a job/task inventory; and c) a general activi- 
ties questionnaire (which serves as a second source of work requirement 
data). The data derived from the three questionnaires are referred to 
in the data bank, respectively, as Forms 02, 03, and 04 data. Copies 
of the three questionnaires may be found in Volume V ir the series 
of volumes reporting the Oregon Studies. 

p rocedurally , the general project questionnaire was administered 
differently than either the job/task inventory or the general activi- 
ties questionnaire. This was a function of the different purposes 
served by the data from the different questionnaires. By and large 
the general project questionnaire was used for purposes of decision mak- 
ing relative to the inclusion or exclusion of a particular project in 
the sample of projects to be described in the Oregon Studies. Given 
this purpose, the Form 02 questionnaire was completed prior to the pro- 
ject site visit. A number of the items in the questionnaire were com- 
pleted at the time that a member of the Oregon Studies staff first con- 
tacted the project director by telephone relative to participation in 
che Studies. During the initial site visit to the project, which was 



designed to make firm a project inclusion-exclusion decision, the pro- 
ject questionnaire was either completed, or, if the project was to be- 
come a participant in the Oregon Studies, it was left to be completed 
by the project director prior to the on-site visit by the data collec- 
tion team. The questionnaire sought such information as the official 
project title, starting and ending dates for the project, the nature of 
the organization within which the project rested, funding sources for 
the project, total funds available to the project, the number and kind 
of personnel associated with the project, and bibliographic references 
to any published or nonpublished documents emerging from the project. 

The job task inventory (Form 03) and the general activities question- 
naire (Form 04^ were administered by members of the data collection team 
while on site by means of a standard procedure. The on-site visitation 
by the data collection team called for a meeting of all staff .n the pro- 
ject during the first morning of the site visit. At this meeting the 
nature of the activities and procedures to be followed by the data col- 
lection staff while on site were outlined, the nature of the demands 
that would be placed upon project staff for purposes of data collection 
were discussed, etc. Typically at the ena of this general orientation 
and discussion period, each staff member was given copies of the Form 
03 and Form 04 questionnaire s and asked to have them filled out by the 
time of the data collection interview. Both questionnaires were picked 
up at the time of that interview, or at least by the time the staff left 
the project site. Of the 1^4 staff that were interviewed 115 completed 
the job/task inventory (Form 03) and 90 completed the general activities 
questionnaire (Form 04). The substantive focus of the job/task inven- 
tory was threefold: professional experience and training, job descrip- 

tion and prerequisites, and supporting services and resources available 
to personnel on the project. The focus of the general activity question- 
naire was, as its title indicates, the activities performed in relation 
to job responsibilities within the project. 



Work Requirement Descriptors 

All data on outputs and on the work related to their production 
were collected through interview. The interview strategy called for: 

(a) identifying O’ puts associated with a project (an output index); 

(b) ordering the outputs according to their interdependencies (an out- 
put map); (c) selecting from the map those outputs for which work re- 
quirement data were to be obtained; (d) identifying persons most directly 
responsible for and/or most directly involved in the production of those 
outputs; and («0 interviewing those persons in relation to the standards 
held for the output being analyzed, the tasks required to produce the 
output, and the knowledges, skills, and sensitivities needed to perform 
the required tasks. The selection of outputs to be analyzed was done by 
the data collection team, on site, after an output map had been estab- 
lished and a sense had been gained as to the outputs that were most 
critical to the project. Persons interviewed provided information rela- 
tive to their own contribution to the production of a particular output , 
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as well as to the contributions of others (a distinction between sell- 
other data was maintained throughout the project.) As familiarity with 
a project grew, adjustments were made as needed in the output map, the 
selection of outputs to be interviewed around, and the matching c>f in- 
terviewees with outputs. All interviews were tape recorded, and all 
data were reduced from the recordings by the person who did the inter- 
viewing . 

The reduction of the interview data involved a multlstep process: 

(£) editing tapes to identify data statements within them, that is, 
statements pertaining to standards, tasks, and enablers; (_b) the re- 
capitulation, or "recapping", of data statements into s readable, gram- 
matically correct form, that is, independent clauses and/or sentences 
(care was taken not to destroy the original language of the interviewees 
in this process); (c) the transfer of the recapped statements to color- 
coded summary sheets that corresponded to the various data sets being 
used; (d) the coding of the recapped statements by a two person coding 
resolution team (during this process the coding team was free to call 
upon members of the data collection team for statement clarification, 
interpretation, context building, etc.); and (e) the storage of the 
coded data in computer files in a way that permitted the interdepend- 
encies within the data to be maintained. A record of all steps in the 
data collection and reduction process was maintained from the time of 
first contact with a project until all data on that project had been 
computer stored and verified. 



The Trustworthiness of the Data 



Since the classes of data reported in the profiles were collected 
by various means, each must be considered separately as to its trust- 
worthiness. Accordingly, the potential sources of error that reside 
within each data class, and the steps taken to control them, are review- 
ed in the paragraphs that follow. 



General Project Descriptors 

Four of the five data sets used to describe the general character- 
istics of projects made use of working documents. These included pro- 
ject objectives, time lines, organizational structures, and context 
maps. Typically, the document used had been prepared by project direc- 
tors. To the extent that such documents can be accepted at face value, 
and to the extent that the Oregon Studies staff did not introduce error 
in reporting the substance of those documents, the data sets that made 
use of them were subject to few sources of error. As a consequence, no 
formal measures of trustworthiness were prepared for then. 

Judgments relative to the trustworthiness of the data reported on 
project dynamics is another matter. It will be recalled that these data 



consist of the pooled observations, hunches, "insights," and choice tid- 
bits of information gleaned by members of the data collection team from 
a wide variety of sources. It will also be recalled that these data 
intentionally were to be subjective and impressionistic. As a means of 
reducing gross error all final descriptions of the dynamics of projects 
were read and confirmed by all members of the data collection team that 
visited a project, but no formal measures as to the trustworthiness of 
such data were obtained. For purposes of profile presentation, however, 
the data on project dynamics are reported. The project directors of the 
last five projects visited were asked to rate and comment on the repre- 
sentativeness of various aspects of the profiles. Four project direc- 
tors indicated the profiles represented a "majority of the operational 
concerns . " 



Personnel and Work Activity Descriptors 

Since the data sets describing personnel and work activities were 
derived through, questionnaire methodology they were subject to the var- 
ious sources of error known to operate within that methodology, for ex- 
ample, the error that is introduced through the selection of questions 
asked, the possibility of multiple interpretations of those questions, 
and the lack of opportunity to determine falsification or shoddiness of 
response to the questions. The steps taken to control these sources of 
error were of two kinds: (a) reasonable care in the development and 

testing of the questionnaires prior to their utilization for purposes 
of data collection; and (b) the administration of the questionnaires 
while the data collection team was on site. The first step involved a 
number of lield trials of the questionnaires, and a number of revisions 
in them on the ba3is of those trials. The second allowed the question- 
naires to be introduced within the context of the data collection effort 
as a whole, and provided within that context an opportunity to clairfy 
troublesome questions about or within them. In combination, it is be- 
lieved that these procedures sufficiently reduced the typical sources 
of error that enter the collection of questionnaire data such that the 
data reported can be viewed with a high degree of confidence. 



Output and Work Requirement Descriptors 



Just as the personnel and work activity data were subject to the 
errors typically associated with use of questionnaires, the output and 
work requirement data, which were collected through interviews, were 
subject to the errors typically associated with interviews. Four sources 
of error have commonly been troublesome in this regard: (a) the selection 

of interviewees as data sources; (b) the information elicited from in- 
terviewees about work requirements; (c) the coding of the information 
obtained from interviewees; and (d) the storage, retrieval, and analysis 
procedures used in manipulating i.he coded data. The procedures followed 



3 When the profiles are being considered as a data base for cross-project 
analyses, other sources of error must be considered. Two critical sources 
are (a) the adequacy of the sample of projects drawn and (b) the adequacy 
of the sample of outputs selected or analysis within a given project. (Coat'd. 
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in the Oregon Studies to combat these sources of error are summarized as 
follows : 

• Interviewee Selection 

Only staff intimately acquainted with or involved in the pro- 
duction of an output were selected for interview. The rela- 
tionship of the interviewee to an output was always confirmed 
by the project director, the person to be interviewed, and 
the immediate supervisor of that person. Data reported by 
an interviewee on the work of others in relation to an output 
were noted and coded separately. 

• Data Generation 

A structured interview procedure was used to obtain data on 
the standards, tasks, and enablers associated with a parti- 
cular output. In the interview, standards were the first to 
be identified, followed by the tasks engaged in to produce 
the output to those standards, followed by the knowledges, 
skills and sensitivities drawn upon in carrying out the tasks 
identified. Stylistic variations in interviewing were per- 
mitted so as to accommodate either interviewer or interviewee 
differences, but during the course of an interview all data 
sets were exhausted. (For a detailed discussion of interview 
procedures see Volume V of the series of volumes reporting 
the Oregon Studies). 

• Data Reduction 



A carefully established set of procedures and decision rules 
were followed in "recapping" the interviewee statements, and 
in coding the recapped statements in terms of appropriate data 
sets. The recapped statements were first checked for their 
completeness and adequacy by the coordinator of data handling 
upon the return of the data collection team from a project 
site. They were checked again by the coding resolution team. 
Incompleteness, or error, or lack of clarity detected on 
either of these checks required that the recapped statements 
be revised until they were acceptable at both quality assur- 
ance checkpoints. To insure reliable coding, team coder agree- 
ments were calculated. Using the recapped statements in three 
case profiles as a base for calculating coder reliability, and 
separating first and second codings by a three month period, 
coding agreements for items in each data set, with one ex- 
ception, ranged between .55 and 1.00. Detailed coder re- 
liability data are reported in Table 4.8. 



These are sources of error that relate to the generalizability of data, 
however, and are not of primary concern in considering the case profiles 
ns descriptions of Individual projects. 
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f > j<J o r Agreements un a Sample of Three Projects, 
With a Period of Three Montha Separating First 
and Second Codings* 



Category 


Category Label 


After Analyzing 
5 Projects 




After Analyzing 
20 Projects 




Symbol 




Project it 1 
N % Agreement 


Project it 12 
N % Agreement 


Project // 13 
N % Agreement 


A 


Structure of Outputs 


20 95.0% 


15 


93.3% 


18 


100.0% 


a 


Function of Outputs 


Not in the 
System 


15 


86.7 


18 


83.3 


B 


Level of Outputs 


Not in the 
System 


15 


71.4 


18 


100.0 


C 


Character of Outputs 


20 95.0 


15 


93.3 


18 


100.0 


Z 


Character of the Focal Output 
to Which Other Outputs Are 
Linked 


Not in the 
System 


15 


80.0 


18 


100.0 


DE 


Clusters of Outputs 


Not in the 
System 


15 


80.0 


18 


88.8 


FGHI 


Primary Categories of Outputs 


20 90.0 


15 


80.0 


18 


60.1 


J 


Structure of Standards 


20 90.0 


37 


100.0 


15 


86.6 


LM 


Primary Categories of Stand- 
ards 


20 65.0 


37 


70.3 


15 


86.6 


NO 


Cluster of Tasks 


23 82.6 


135 


/ 9.0 


50 


80.0 


QR 


Primary Categories of fa*ks 


23 52.2 


135 


54.8 


50 


72.0 


S 


Structure of Enablers 


38 92.1 


114 


100.0 


35 


100.0 


UV 


Primary Categories of Enablers 

i * 

\ 


38 65.8 


114 


69.3 


35 


77.3 



* All coding in the Studies was carried out by a two man team. Each member of the 
team coded all items independently, compared codings, and then resolved differences 
when differences occurred by reference to the taped interview, discussion with the 
interviewer, etc. The reliability data reported are for coding agreements reached 
by the team. 
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9 Data Storage and Retrieval 



A 3 soon r 3 the recapped statements had been coded for a parti- 
cular project the codes were forwarded for transfer into com- 
puter storage. After storage, repeated checks were rui to 
insure that the initial comp iter entries were correct, and 
the computer center manipulations over time had not destroyed 
or reordered the data as it was originally stored. 

In view of the data that have been presented on the reliability 
of coding, and the precautions that were taken throughout the data col- 
lection effort in instrumentation, collection, and reduction, the data 
reported in this and other volumes can be viewed with a reasonable degree 
of confidence. 



The Organization of the Data 



Four organizing frameworks have been used in presenting the data 
collected in the Oregon Studies: a) projects; b) the focus of projects; 

c) the outputs of projects; and d) selected issues. The rationale for 
organizing the data according to these frameworks an overview of the 
data presented within them, and directions as to where the data so ordered 
are to be found are provided in the paragraphs which follow. 



The Organization of the Data By Projects 

The organization of the data by projects meets the contractual 
obligation of the Oregon Studies to present the data collected in case 
study form. Accordingly, "case profiles” are presented for each of 
the 20 projects studied. All data collected on each project are inte- 
grated and presented within the profile of that project. In the explora- 
tion and mapping language adopted for reporting the work of the Oregon 
Studies, the case profiles are presented as detailed "site” maps. As 
such, the profiles collectively constitute one of the primary data out- 
puts of the Oregon Studies. 

The profiles are reported in Volume IV in the series of volumes 
reporting the Oregon Studies. Part One of Volume IV contains five 
profiles that describe research projects and three that describe eval- 
uation projects; Part Two of the Volume contains seven profiles that 
describe development projects; and Part Three of the Volume contains 
five profiles that describe diffusion projects. The rationale for the 
development of detailed case profiles of ongoing RDD&E projects in ed- 
ucation at this point in time has been reviewed in Chapter 1 of the 
present volume. A description of the content and organization of the 
profiles appears in the following chapter. 
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The Organization of the Dat a By Focub 

The organization of the data by project focus meets the contractual 
obligation of the Oregon Studies to M map the domain" of educational RDD&E. 
Three major classes of data have been attended to in this mapping process: 
a) descriptors of project personnel; b) descriptors of project outputs; 
and c) descriptors of work requirements associated with the production 
of project outputs. In all three cases the data are reported in terms 
of the range of phenomena observed within the total sample of projects 
studied, and the similarities and differences in the distribution of 
those phenomena when ordered according to the focus of the projects 
studied, that is, when ordered according to RTD&E. In the exploration 
and mapping language adopted for reporting the work of the Oregon Studies, 
these dat*\ are presented as preliminary or "outline" maps of the domain 
as a whole. These would seem to be appropriate labels for such a map- 
ping effort for -hey are based upon a relatively small project sample 
(20) and an unequal distribution of projects by RDD&E within the sample.^ 

The data ordered by project focus are reported in Chapters 6, 7, 

8, and the appendices of the present volume. Chapter 6 contains the data 
on personnel associated with educational RDD&E activities; Chapter 7 con- 
tains the data on the nature of the outputs that emerge from educational 
RDD&E activities; and Chapter 8 contains the data on the standards set 
for those outputs, the tasks involved in their production, and the know- 
ledges, skills, and sensitivities that are drawn upon in producing them. 
The full range of outputs, standards, tasks and enablers observed in 
the 20 projects studied appear in the appendices. Each class of data 
reported in Chapters 6, 7, and 8 are reported as if they were independent 
sets; they are reported as linked sets in Chapters 9, 10 and 11. 

The Organization of the Data By Outputs 

The organization of the data by outputs meets the contractual 
obligation of the Oregon Studies to "identify the competencies needed 
by staff to carry out the operations involved in educational RDD&E pro- 
jects, the indicators acceptable as evidence of the existence of such 
competencies, and the knowledges, skills, and sensitivities prerequisite 
to their demonstration". Three sets of data are presented in this regard: 
a) data on outputs per se ; b) data that link standards, tasks, ^nd en- 
abling knowledges, skills, and sensitivities separately to outputs; and 
c) data that links standards, tasks, and enablers collectively to outputs. 



^ As pointed out: elsewhere, the data base provided by the Oregon Studies 
for describing the range of personnel, outputs and work requirements that 
appear within the domain of educational RDD&E is leasouably strong, whereas 
the data base provided by these Studies for the analysis of similarities 
and differences between educational RDD&E projects is relatively weak. At 
best, the similarities and differences data can be only suggestive. Pre- 
ferably, it should be treated as a source of hypothe as to be tested in 
subsequent studies . 
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In the exploration anl mapping language adopted for reporting the work 
of the Oregon Studies, thase data are presented as "suggested detail 
maps". This would seem c ) be an appropriate label for such a mapping 
effort for it is based on a relatively large sample of outputs (962 
identified; 298 analyzed for their work requirements), and work require- 
ment data are reported only for those outputs that appeared most fre- 
quently within the projects studied. 5 

As indicated above, the data ordered by outputs are reported ir 
Chapters 9, 10 and 11 in the present volume. Chapter 9 contains th^ 
dat^ reported on outputs per se ; Chapter 10 contains the separately 
linked output - standards data, output - task data, and output - en- 
abler data, and Chapter 11 contains illustrations of aJl three classes 
of work requirement data linked to a single output. The concept of 
an "output tree", as illustrated in Chapter 11, is seen a 3 being the 
most powerful organizing framework for reporting work requirement data 
that is presently available. It is presented in Chapter 11 as an il- 
lustration because in actual operation an output tree represents only 
a collective reporting of the data summarized separately in Chapter 10. 



The Organization of the Data By Selected Issues 

The organization of the data by selected issues meets the respon- 
sibility of the Oregon Studies to determine the extent to which the 
factors of project size and setting (the two factors used in selecting 
projects for study, in addition to project focus) account for variability 
in the data, and the responsibility to apply the data base established 
in the Studies to questions of high interest to trainers and others 
responsible for the conduct of educational RDDaE. Accordingly, four 
additional analyses of the data are presented; a) an analysis of the 
variability associated with project size; b) an analysis of variability 
associated with project setting; c) an analysis of variability associated 
with job role; d) an analysis of variability associated with output - 
work requirement data and job role; and e) an explanation of the variety 
of combinations of the data sets that are possible. In the exploration 
and mapping language adopted for reporting the work of the Oregon Studies, 
these data are presented as "special purpose" maps. All special purpose 
maps are presented in Chapter 12 of the present volume. 



^ All work require ient data were collected, coded and computer stored 
in terms of the r linkage to outputs, so it is possible to prepare sui 
maries of work requirement for all 298 outputs analyzed. Since these are 
reasonably costly data to retrieve and present, however, ana since a large 
number of the outputs analyzed had reasonably small frequencies of appear- 
ance, the decison was made to undertake at the present time only a limited 
number of such analyses. It is the opinion of the Oregon Studies staff, 
however, that these are the most powerful and useful cross project data 
reported in the Studies and that further analyses of this kind should be 
undertaken using the existing data base. Until such analyses are under- 
taken, however, or if they are not, the reader interested in the linkage 
of work requirement data to output* is directed to the case prof Lies 
where such linkages are reported for each output analyr^d. 



Chapter 5 



SITE MAl^S : PROFILES OF ILLUSTRATIVE PROJECTS 



The primary units of data to emerge from the Oregon Studios are 
the twem y case profiles contained in Volume IV of the series reporting 
the Studies. These "site maps" describe five educational research proj- 
ects, seven educational development projects, five educational diffusion 
projects, and three educational evaluation projects, all of which were 
judged to be of a quality illustrative of desirable future activity. 1 
Each individual profile contains data representative of all the classes 
of data described in the preceding chapter, i.e., general project de- 
scriptors, people descriptors, and conceptually lirked descriptors. 

The purpose of the present chapter is to introduce the reader to 
profile organization and content, to show how the variables described 
in Chapter 4 are included in profiles, and to discuss the utility of 
the profiles for students, trainers and practitioners. For additional 
detail not contained within the present chapter, the reader is referred 
to the sections NOTES ON THE DEVELOPMENT OF THE PROFILES and A GUIDE TO 
READING THE PROFILES contained in Volume IV. 



Profile Organization and Content 



A profile is organized around seven major components or chapters. 
These are: 



Chapter I 
Chapter II 
Chapter III 
Chapter IV 
Chapter V 
Chapter VI 
Appendix 



Overview 

Parameters of the Project 
Summary of Output Data 
Supplementary Data 
Project Dynamics 
Implications for Training 
(Recappea Interview Data) 



Included in the rationale for a profile is the assumption that data 
collected from a site has its greatest meaning when displayed in the 
context from which it was derived. Thus the initial chapters move from 
general project descriptors, to people descriptors, and finally to 



1 In the Oregon Studies the definition of "project" allows for the 
inclusion of ongoing activities such as public school activities in 
which it is possible to specify the parameters of those activities in 
terms of identifiable ends and support bases. 
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conceptually linked descriptors. Closing chapters extrapolate from 
those data and from the impressions of interviewers to capture the 
essence and nature of the realities of a project. The folic wing para- 
graphs provide overviews of each chapter, indicate the location of the 
data sets descriuad in the chapters, and describe the way the data sets 
are assembled. 



Profile Chapter I: Overview 

Chapter I consists of three standard parts, which provide t. nr > bulk 
of the basic project descriptors, each of which deals with various as- 
pects of the project conLext. The parts are: 

(a) A synopsis of the project; 

(b) The objectives, rationale, and significance of the project; 
and 

(c) The context in which the project operates. 

The synopsis (section a) is simply a standardized one page listing of 
descriptors similar to those provided on the cover sheet of a proposal. 
In effect it identifies che project in terms of the responsible agency 
and officers, the level of funding, the duration, the number of staff, 
etc. Section (b) further describes the project in terms of main 
thrusts, underlying assumptions and/or procedures which guide it, and 
the fit of f T \e project in any larger scheme of things. The section, 
which is derived from project abstracts and interviews, makes more 
explicit the intended target of the effort and the consumers expected 
to have an interest in it. The context section (c) provides a Svimmary 
of the time, place, and circumstances of the project. Typical dimen- 
sions, derived from observation as well as documents and interviews, 
include descriptions of the project’s: 

(1) relationships to other agencies and groups, 

(2) substantive relationships to other, larger programs, 

(3) support and technological resources, 

(4) time dimensions and related constraints, and 

(5) physical and environmental setting. 

Central to the overall discussion of a project’s context is a "con- 
textual map" which illustrates the significant, interacting aspects of 
the environment. 



Profile Chapter III Parameters of the Project 

The second chapter of a profile contains data on people and on 
conceptually linked descriptors organized into four standard parts: 

(a) staff structure 

(b) project roster 

(c) index of outputs 

(d) output map. 

Those elements describing people are found in the staff structure and 
roster sections. The discussion of staff structure is aided by a 

<s 
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chart which essentially illustrates the functional relationships among 
positions held, and/or roles carried out, by people in the project. 

The degree to which the i^ustrafion does, in fact, demonstrate func- 
tional relationships, rathe, than another level of "organizational 
structure/ 1 is dependent on the philosophy of the project being de- 
scribed and on the way personnel spoke about the structure. Project 
roster data includes a listing of the positions held by personnel and 
includes annotations which more fully describe the responsibilities to 
the project of the people holding each position. Botl staff structure 
and roster data were derived from observation, documents, and inter- 
view, and were confirmed by project personnel as being accurate repre- 
sentations . 

Chapter II introduces the first of the conceptually-linked de- 
scriptors. In this chapter a confirmed, anno^ted index of project 
outputs derived from observation, project doc. its, and interviews 
is presented. This is followed by an "output ’ which illustrates 
the interdependencies of the outputs indexed, irrespective of time 
frames. While not considered an exhaustive listing of all possible 
outputs of a project, the outputs displayed are considered to be 
representative of the major concerns of the project. This means that 
outputs sampled cover the range of outputs which, in terms of struc- 
ture, character, focus, and function, as conceptually defined in 
Chapter 4, led lo a description of project operations which captured 
the essence of the project as indicated by project personnel. The 
indexing and mapping strategies provide a comprehensive view of the 
kinds of outputs a project produces, facilitate an understanding of 
the complexity of the operations, and contribute to the provision of 
equivalent data across profiles. 



Profile Chapter III; Summary^ of Output Data 

The preceding profile chapter flows into Chapter III and allows 
it to extend the conceptually linked descriptions of outputs. It is 
in Chapter III that the work requirements which relate tc each of a 
selected sample of outputs indexed for a project a e reported. Stand- 
ardized sections of this chapter include: 

(a) standards held for outputs 

(b) tasks pertaining to output attainment 

(c) enablers pertaining to output attainment. 

The data presented are derived from interviews focusing on each 
of the outputs selected ^or analysis. These interviews have been 
recapped and coded into empirically derived category sets (see Appen- 
dices 1 to 9 in this volume) which describe these variables in man- 
ageable form. Tables of data linking these empirically derived de- 
scriptors to specific outputs are provided and serve as the focus of 
discussion. Narration around the tables provides a basis for inter- 
pretation of the data shown within them and highlights the data that 
are to be discussed in subsequent profile chapters. 

The data tables discriminate the structural elements of stand- 
ards and enablers, and the cluster elements of tasks, as is shown in 
Figure 4.1 in Chapter 4. Following these tables, and a narrative 






description of each, a general discussion of any significant inter- 
relationships between them is- offered. 



Pr ofile Chapter IV: .Supplementary Data 

Chapter IV is not as highly structured as the preceding chapters 
in the profile. It presents data on people and on conceptually linked 
descriptors and highlights those which appear to have particular 
relevance to an adequate description of a particular site. Standard 
elements include (not necessarily in the following order): 

(a) frequencies of outputs (all those indexed) as ordered by 
structure, character, focus, and function; 

(b) selected interviewee responses to questionnaire items 
relating to position requirements, support resources, and/or 
pr o j ec t managemen t ; 

(c) interviewee responses to questionnaire items relative to 
the comparative emphasis given various classes of work 
activities . 

Tables of data are presented whenever they serve to provide a neces- 
sary basis for discussion. Narrative discussion may include a com- 
parison of these data with interviewer observations and/or with the 
data contained in other chapters of the profile. 



Profile Chapter V: Project Dynamics 

The fifth chapter of a profile focuses on those general project 
descriptors having to do with project dynamics. While these data were 
collected in a less systematic way, "cue sheets" served to sensitize 
the interviewers to a variety of factors which might be observed on 
a site. In addition, the cue sheets provided a systematic means for 
the profile writer to "debrief" the team and gain access to the pooled 
perceptions of the team. 

This chapter, by design, is the least structured of the profile 
chapters. Its purpose is to "round out" the profile by reporting 
impressions about the site which the data collection team gained by 
virtue of having "lived" on the site for a number of days. The im- 
pressions reported are presented in whatever sequence, form, and 
substance the writer considered best in calling out the significant 
and unique features which guided the functioning of the site. This 
freedom to vary was considered essential to extending the meaning and 
credibility of the data collected. At the same time it provided a 
vehicle for identifying new classes of data which might be deserving 
of quant if ication . 

The elements of project dynamics to be found in the collective 
profiles derived from, but were not limited to, the observable and 
"felt" influences on projects, and the people within them, as follows : 
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(a) staffing patterns 

(b) management structure 

(c) management styles 

(d) commitments 



(e) affective elements 

(f) issues 

(g) staff backgrounds 

(h) agency interrelationships 



This list is representative and not meant to be inclusive. Discussion 
of these elements is linked, when appropriate, to the data contained in 
previous chapters as a means of illustrating any existing characteris- 
tics of consistency or cohesiveness in the project. In keeping with 
the intent of the profile, however, such discussion is not meant to be 
interpreted as an evaluative statement by the Oregon Studies staff. 
Rather, its purpose is to point up features which appear to be exem- 
plary instances of a phenomenon or a set of problem-solving or issue- 
resolving behaviors. 

Profile Chapter VI: I m plications for Training 

In the final chapter of the profiles the reader will find a sum- 
mary discussion of various elements of training having specific rele- 
vance to the project being described. Because the basic profile data 
is considered the primary source of information regarding the content 
of training, this summary generally comments only briefly on the basic 
data and elaborates on any strategies or elements which were recom- 
mended by project personnel or were implied by the nature taf the data 
collected. Frequently the training needs mentioned by projects were 
a function of recalling difficulties with one or more problems en- 
countered by the project. To a certain extent, then, it can be assumed 
that those items mentioned may highlight areas where competence was 
needed and in which preparation was weak. 

Profile Appendix 

The Appendix to a profile contains statements derived from tape 
recordings of interviews with personnel. These statements represent 
the "raw data" base for the conceptually-linked descriptors reported 
in Chapter III of the profile, i.e., the standards, tasks, and ena- 
blers which relate to the generation of specific outputs. The inclu- 
sion of the appendices in the profile serves two essential purposes, 

(a) to give substantive meaning to the category labels used to sum- 
marize the data reported in Chapter III, and (b) to provide the ini- 
tial basis for refining and ordering data of significance to trainers 
in developing output oriented skills. All data analyses dealing with 
outputs and work requirements in the following chapters of Volume I 
are based upon these raw data statements. 



The utility of a profile varies according to various elements 
having special relevance for different audiences. Considerations of 
utility must be tempered by an awareness of the strengths and 



The Utility of a Profile 
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weaknesses which may be inherent in the data residing in the profiler. 
Tin* following paragraphs suggest possible uses of profiles. 

One of the more obvious and unique contributions a profile can 
make is as an instrument of training through its description of "what 
life is like" within the context of an ongoing educational RDD&E proj- 
ect. From the nature of its outputs through the nature of the people 
producing them to the elements of interaction which take place, the 
user gets a sense of how things come together. The lessons learned 
by the participants in a project are highlighted and become, in and 
of themselves, useful elements of information. 

Taken one al a time, the weaknesses of a profile revolve essen- 
tially around the fact that no single profile (nor any single project 
for that matter) illustrates the range of all possible alternatives 
that might have been pursued in achieving a given purpose. Even in 
the collective sense, by virtue of the variability sought in the proj- 
ects studied, expansion of the possible alternatives relating to a 
single purpose is not easily accomplished. The profiles as a set, 
however, do afford the user with 20 instances of what life is like 
within projects pursuing 20 essentially different purposes. To that 
extent, the profiles become rich sources of descriptive Information. 

In short, whether the user be a student attempting to become 
acquainted with the field, a person just entering the field, or a 
project director or policy maker thinking through the kinds of prep- 
aration necessary to carry out a particular kind of project, the pro- 
files illustrate the orchestration of various elements into an inte- 
grated unit. What is found to exist in such an integrated unit is 
perceived to be the essence of the focus of training. The profile 
attempts to put the "meat" on the bones of the "body" that is educa- 
tional RDD&E rather than providing only a view of the skeleton. The 
how and why of a series of directed activities were the target of the 
profile. In finding the how and why, the attempt was made to capture 
the human element as it relates to the kinds of things produced. Each 
profile is unique in this sense, representing not only different 
people, but different contexts and problems. The reader is encouraged 
to review the profiles from the "human" side, and to seek evidence of 
the process of the coming together of people to solve a problem. 



Chapter 6 



OUTLINE MAP A: PERSONNEL WITHIN EDUCATIONAL RDD&E 



It will be recalled from reading Chapter 4 that this is the first 
of three chapters chat organize the data collected in the Oregon Studies 
around the focus of the projects studied, that is, around research, 
development, diffusion, and evaluation. It will be recalled also that 
the intent of these three chapters was twofold: (ja) to provide a picture 

of the range of personnel, outputs, and work requirements found within 
the domain of educational RDD&E; and (b_) to provide a picture of the 
similarities and differences within those data when they are ordered by 
RDD&E. Finally, it will be recalled that in these three chapters all data 
are reported as if they were independent data sets. Personnel are described 
apart from the projects in which they worked; outputs are reported as if 
they were independent of the context in which they rest; and work require- 
ments are reported as if they were independent of the outputs to which 
they are linked. The purpose of Chapters 6, 7, and 8, therefore, is simply 
to provide a sense of the kind of personnel, outputs, and work require- 
ments that exist within the domain of educational RDD&E, and how they are 
distributed within that domain. The interdependencies between these data 
sets are reported in subsequent chapters. 

The aim of the present chapter is to describe the personnel working 
within the 20 projects studied. Data about personnel are organized to 
describe: ( a ) identifying characteristic a; (b^) educational experience; 

(c_) work experience; (d^) services and equipment relied upon; and ( e ) 
supplementary observations. As background information, the contractual 
relationship between the personnel and the projects studied, and the roles 
personnel performed within those projects, are also described. All data 
reported in the present chapter, with the exception of trie supplementary 
observations, were collected through quea tionnaires administered while 
the data collection team was on-site. 

One hundred and thirty-four persons were interviewed during the 
course of the Oregon Studies. Of these, 115 completed the questionnaire 
from which the data reported in the present chapter were derived. Since 
the relationship of the personnel interviewed to the projects in which 
they were working needs to be considered in interpreting these data, some 
effort must be made to understand this relationship. Two sets of data are 
reported toward this end: (a^) a description of the contractual relation- 

ship of the interviewees to the project studied, and (b) a description of 
the roles they performed. The first set of data is summarized in Table 6.1. 
The second set is summarized in Table 6.2. 



The Contractual Relationships Between Personnel and Projects 



It will be seen from Table 6.1 that a variety of contractual relation- 
ships were held by personnel on the 20 projects studied. Approximately 
25% of the 115 personnel queried held positions other than regular staff 



membership. These ranged fro* a subcontractor relationship to being a 
member of the staff of user agencies. Contractual arrangements between 
staff and projects were not coaparable across RDD&E, however. Relatively 
few staff interviewed In research projects had other than regular staff 
appointments, while a relatively large number of Irregular staff arrange- 
ments (approximately 1/3) were found within development, evaluation, and 
diffusion projects. It will be noted also that more staff nembers in 
development, diffusion, and evaluation projects were interviewed than in 
research projects. This tends to reflect differences in staff size gener- 
ally observed withlu sach projects. 

TABLE 6,1 

The Contractual Relationship of Interviewees 
With the Projects Studied 



Relationship 
to Project 




Project 


Focus 




Total 

(20) 


Research 

(5) 


r 

Development { 
(7) | 

I 


Diffusion 

(5) 


Evaluation 

(3) 


Regular staff member 


16 


34 


24 


15 


89 


On-call staff member 


1 


— 


3 


— 


4 


Regular subcontractor 
staff member 


— 


4 


1 


— 


5 


On-call subcontractor 
staff member 


— 


1 


— 


— 


1 


Advisor / consul tan t 


— 


2 


1 


— 


3 


Manager/ special 1st 


— 


1 


4 


4 


9 


Member of partici- 
pating agency 


— 


3 


— 


— 


3 


Member of user agency 


— 


— 


— 


1 


1 


Other 


— 


— 


— 


— 


— 


TOTALS 


17 


45 


33 


20 


115 
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The Roles Performed by Personnel Interviewed 



To obtain further clarity as to the nature of the personnel sampled, 
each person responding to the questionnaire was assigned a project role 
classification. A nine category classif iation scheme was used. Place- 
ment within this scheme was arrived at through face-to-face discussions 
with each respondent to the questionnaire. The distribution of the 115 
respondents by Job roles is summarized in Table 6.2. 

TABLE 6.2 

The Project Roles Performed by the Interviewees 







Project Focus 




Total 

(20) 


Project Role 


Research 

(5) 


Development 

(7) 


Diffusion 

(5) 


Evaluation 

(3) 


Principal Investi- 
gator 


5 


5 


2 


2 


14 


Project Director, 
other than Princi- 
pal Investigator 


2 


5 


4 


1 


12 


Top level project 
manager, other than 
Project Director or 
Principal Investi- 
gator 


2 


11 


6 


6 


25 


First-line Super- 
visor 


— 


— 


2 


3 


5 


Member, professional 
staff 


5 


19 


16 


8 


48 


Member, clerical 
support staff 


1 


— 


— 


— 


1 


Member, technical 
support staff 


2 


— 


2 


— 


4 


Advisor or consultant 


— 


5 


1 


— 


6 


Other 


— 


— 


— 


— 


— 


TOTAL 


17 


45 


33 


20 


115 
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In studying thin table it la Intriguing to note that approximately 
IwiJt of the per.sona interviewed Identified thamaelvee aa holding poaltlona 
lu tlu* top four 'integer lea which deecrlbc "management" type roles. Exclud- 
ing advisors and consultants, 51 respondents classified themselves as 
"managers" while only 53 classified themselves as staff members. With the 
exception of diffusion, each of the project focus areas actually had more 
of the persons interviewed in management level roles than in regular staff 
roles. While such data can be accounted for partially by interviewee 
selection factors, they nevertheless raise interesting questions about who 
does what work on RDD&E projects. 



Identifying Characteristics of Educational RDD&E Personnel 

Table 6.3 presents data collected around a number of characteristics 
that identify RDD&E personnel. It Includes data on sex, age, degree held, 
number of publications, salaries, and professional membership. These data 
should serve to give the reader some notion of the general characteristics of 
personnel found within the field. It should also provide the prospective 
RDD&E staff member with a sense of the work world into which he is moving. 

As Table 6.3 shows, the majority of personnel in educational RDD&E are 
male. This is the case by a margin of nearly two to one. A few respondents 
indicated ages between 20 and 24 years, and three were over 54, but the 
majority fell in the age range of 25 to 44. Respondents held degrees rang- 
ing from bachelors to doctorate or higher, but all were college graduates. 

No one interviewed held other than academic degrees at the bachelors level or 
beyond. It is interesting to note that research personnel, while younger 
than development personnel, were more likely to possess doctorates. 

While college graduation appears to be an essential characteristic for 
work within the context of educational RDD&E, frequent publication does not. 
Fifty-seven of the 95 interviewees who chose to respond to the question of 
publications indicated that they had published anywhere from one to more 
than 16 ar tides , - but 38 indicated that they had not published at all. Of 
note is the large proportion of personnel in the areas of research, develop- 
ment, and diffusion who cited no publications. 

Salaries for RDD&E personnel ranged quite dramatically from below 
$5,000 to above $29,000, with all ranges represented in between. 

At least 12 different professional associations could claim RDD&E 
personnel as members. While AERA was the most frequently cited, RDD&E per- 
sonnel apparently benefited from membership in such diverse associations as 
NEA and the Psychonomic Society. 

In reviewing the data presented in Table 6.3 by project focus, a 
number of interesting trends can be identified. In terms of sex, for 
example, it appears that diffusion projects are the only ones where the 
male-female ratio favors females. Research, development, and evaluation 
projects appear to be dominated numerically by males. With respect to 
age, development projects appear to attract personnel over 40 while 
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TABLE 6.3 



Identifying Characteristics of Project Personnel Interviewed 



Variable 

Subclass if icat ions 




Project Focus 




i u t a 1 
(20) 


Research 


Development 


Diffusion 


Evaluation 




(5) 


(7) 


(5) 


(3) 





Sex 



Male 


13 


33 


15 


15 


76 


Female 


4 


12 


18 


5 


39 


Age 


No Response 


2 


3 


5 


— 


10 


Under 20 


— 


— 


— 


— 


— 


20-24 


1 


2 


— 


— 


3 


25-29 


3 


6 


5 


6 


20 


30-34 


4 


7 


4 


2 


17 


35-39 


5 


4 


10 


7 


26 


40-44 


— 


12 


5 


3 


20 


45-49 


1 


5 


1 


1 


8 


50-54 


1 


3 


3 


1 


8 


Over 54 


— 


3 


— 


— 


3 


Degree 


No Response 


— 


4 


— 


— 


4 


Doctorate or 


8 


10 


5 


9 


32 


Post Doctorate 












Specialist or 


1 


1 


— 


— 


2 


Professional 












Masters 


4 


17 


17 


9 


47 


Bachelors 


4 


13 


11 


2 


30 


Associate 


— 


— 


— 


— 


— 


Technical 


— 


— 


— 


— 


— 


License 












None of These 


— 


— 


— 


— 


— 






Number 


of Publications 






No Response 


2 


ii 


7 


— 


20 


None 


8 


13 


12 


5 


38 


1-3 


— 


7 


7 


6 


20 


4-8 


2 


6 


5 


7 


20 


9-15 


1 


2 


1 


1 


5 


16+ 


4 


6 


1 


1 


1 2 
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TABLE 6.3 (con't.) 



itlciiL Lt'yiuK Character Lstica of Project Personnel Interviewed 



Variable 

Subclass if icat ion3 


Project Focus 


JOt.l 1 
(20) 


Research 

(5) 


Development 

(7) 


Diffusion 

(5) 


Evaluation 

(3) 


Salary 


No Response 


— 


5 


1 


— 


6 


Under $5,000 


6 


3 


— 


1 


10 


$5,000-38,999 


1 


3 


1 


— 


5 


S9,000-$ll,999 


2 


8 


9 


2 


21 


$12,000-$16,999 


3 


13 


14 


6 


36 


$L7,000-$21,999 


1 


8 


2 


8 


19 


$22,000-$29,00C 


4 


3 


2 


3 


12 


Over $29,000 


— 


2 


4 


— 


6 



Professional Associations 



AERA 


7 


9 


8 


15 


39 


NEA 


— 


6 


7 


2 


15 


AVA 


— 


1 


— 


— 


1 


APGA 


— 


1 


— 


2 


3 


APA 


4 


1 


3 


4 


12 


AECT 


— 


— 


— 


— 


— 


Other* 


7 


24 


17 


12 


60 


*ALA 







2 





__ 


ASA 


3 


— 


— 


— 


— 


ASCD 


— 


2 


2 


— 





ASIS 


— 


2 


3 


— 


— 


NOME 


— 


— 


2 


2 





NOTE 


— 


7 


— 


— 





Psycho nomic 
Society 


2 











the other three foci are staffed by a majority of personnel who are 39 or 
younger. It is noteworthy, however, that while development project staff 
appear to be older, they hold proportionately fewer doctorates than either 
research or evaluation staff. Only about 1 in 4 development staff had 
acquired doctorates, while in both research and evaluation close to half 
of those responding held doctorates. Diffusion focused projects had the 
lowest ratio of doctorates (about 1 in 6). 
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Salaries tended to favor evaluation personnel, with over half of 
their number (11 of 20) earning salaries in excess of $17,000 yearly. 

Both diffusion and development projects show a wide range of salaries, 
with the largest staff loading falling in the $12,000 to $17,000 bracket. 
For those working on research projects the picture appears a little bleak. 
While four researchers earned upwards of $22,000, «?lx earned less than 
$5,000 per year. Those earning less than $5,000 were mainly graduate 
students, however, so the salaries for full time workers are not as low 
ns the table might suggest. 

The data on professional membership indicate that less than 50% of 
those interviewed belonged to any organization, and of those who did 
claim memberships manv belonged to more than one association. It appears 
that AERA was the most popular msociation for staff in all project foci, 
with a substantial number of diffusion and development personnel holding 
membership in NEA. 

Taken as a whole these data do not present a picture of educational 
RDD&E that quite fits a "traditional" view of the field. There were 
proportionately fewer staff that held doctorates than expected, espec- 
ially considering the high proportion of "manager" level roles filled 
by personnel studied. There were also fewer publications, fewer member- 
ships in AERA, and higher salaries than expected. Contributing factors 
here may include the project settings. Four of the five research projects 
studied, for example, were located in college/university settings. While 
the youth of the research personnel sampled may help to explain the small 
number of publications produced, generalization of such a hypothesis to 
the development, diffusion and evaluation parts of the sample does not seem 
to follow with the same logical ease. Could such factors as the differ- 
entiation of responsibilities within projects, their size and setting 
have contributed? Logically, and from Impressions obtained by the inter- 
view staff, any number of these factors could have played a significant 
role. 



The Educational Experience of RDD&E Personnel 



One of the Important characteristics of RDD&E staff is the educational 
experience they have had. While degrees held are one measure of academic 
preparation, it was felt that data on areas of specialization would also 
be helpful in arriving at a picture of personnel working within the field. 
Accordingly, the major and minor academic fields of study pursued by the 
interviewees while in college are displayed in Table 6.4. 

Major areas of study cited by RDD&E personnel ranged from the predict- 
able Education/Teaching category to such diverse fields as Engineering, 
Business Administration and a sizable number of "others." The diversity 
of minors is equally extreme and, although there is a heavy emphasis on 
education-related fields, the number citing other areas is impressive. 

(A discrepancy will be noted in the totals at the bottom of Table 6.4. 

This is accounted for by the fact that some respondents cited two majors, 
and some cited two or more minors.) 

Within each of the focal areas a number of interesting findings appear. 
Personnel working on projects with an educational research focus cited no 
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majors In (education, and only two majors in Educational Research. The 
heavy emphasis for research personnel appears in the area of Psychology. 
Minors for this group indicate the same trend. Only one of the project 
focus areas, development, shows a substantial grouping of majors in the 
area of Education/Teaching. Even here, however, that Is not the largest 
cluster, for while 11 of the staff of development projects cited Education/ 
Teaci ing as a major, 14 cited areas other than those listed. Even as a 
minor, Education/Teaching did not dominate the educational experience of 
those in the field. 



TABLE 6.4 



The Educational Experience of RDD&E Personnel 



Educational 

Experience 


Major (s) 


Minor (8) 


Totals 


R 


0 


D 


E 


R 


D 


D 


E 


MaJ. 


Min. 


No Response 


1 


4 


— 


— 


7 


16 


9 


2 


5 


34 


Education/Teaching 


— 


11 


5 


5 


1 


9 


5 


4 


21 


19 


Education Administration 


1 


6 


5 


2 


— 


1 


1 


— 


14 


2 


Educational Research 


2 


1 


3 


2 


2 


3 


2 


4 


8 


11 


Guidance /Counsel ing 


3 


2 


1 


2 


— 


— 


2 


1 


8 


3 


Vocational Training 


















“ ““ 




Statistics /Measurement 


— 


— 


1 


2 


1 


3 


3 


6 


3 


13 


Psychology 


7 


1 


4 


5 


4 


4 


5 


4 


17 


17 


Engineering 


— — 


1 








1 




— “ “ 


1 


1 


Computer Sciences 




















”” 


English/Writing 


— 


7 


4 


— 


— 


8 


4 


1 


11 


13 


Business Admin is t rat ion 


— 


2 


2 


— 


— 


1 


1 


1 


4 


3 


Other 


4 


14 


12 


5 


2 


5 


*.0 


7 


35 


24 


TOTALS 


18 


49 


37 


23 


17 


51 


42 


30 





Surprisingly, majors and minors in Psycnology and Engllsh/Wrlting were 
reasonably common in the sample of personnel as a whole. Psychology appeared 
consistently across foci both as a major and a minor. Majors and minors in 
English/Writing seen to concentrate in development and diffusion projects. 
Apparently college experience in a wide range of fields outside education 
has relevance to the work of educational RDD&E. 



The Work Experience of RDD&E Personnel 



In an attempt to describe the work background of KDD&E personnel, 
data weie collected In relation to work settings and wo.-'t experiences. 

Work setting data are reported In terms of 9 categories. Including such 
background as public school experience and research in colleges or uni- 
versities. These data provide information about the contexts in which 
educational RDD&E personnel work. Work experience data are reported in 
terms of six descriptors such as Experience in Educational RDD&E and 
Times as Principal Investigator. Additional data are reported that 
describe the experience demanded or required cf RDD&E personnel to per- 
form the work for which they are currently responsible. 

The data displayed in Table 6.5 indicate the type of settings in 
which the RDD&E personnel interviewed have worked, and the number of 
years spent in those settings. Those responding (many staff either indi- 
cated no previous work experience or did not respond to chese questions) 
cited all nine work setting categories, and indicated having worked in 
the settings anywhere from 0 to 16 years or more. (A number of respon- 
dents indicated having worked in two or more settings for varying lengths 
of time.) The two settings cited most frequently by RDD&E personnel 
generally were public school and present organization. Experience in 
both settings ranged from 1 to 16 or more years. Of interest in this 
regard was the small number of personnel in research who indicated they 
had had work experience in public schools. Only four research personnel 
cited experience in public schools and of those, five years was the maxi- 
mum time spent there. 

Other work settings in which research personnel had worked included 
college research/ teaching and work in R & D centers. A relatively high 
proportion of personnel in both diffusion and development projects indi- 
cated work experience in the public schools. In many cases that experience 
extended beyond six years duration. Evaluation personnel indicated a 
frequent history of experience in college settings and public schools. 

This was somewhat unexpected in that data reported previously have tended 
to show evaluation and research personnel to be similar, but a breed apart 
from personnel in development and diffusion projects. 

In conclusion it can be said that personnel in development projects 
tend to display a history of work experience in a wider range of settings 
than personnel in research, evaluation, or diffusion projects. Personnel 
in diffusion projects follow closely in this regard, and are followed in 
turn by personnel in evaluation and research projects respectively. 

The data displayed in Table 6.6 indicates the on-the-job work exper- 
ience of educational RDD&E personnel. Six categories of work experience 
were the focus of data collection. These categories ranged from general 
RDD&E experience to project management experience. The data are displayed 
to show the responses of personnel to categories of work experience by 
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project f ocufa . The total number of respondents shown by category differ 
because some interviewees did not respond to some of the questions used to 
elicit data in each category. 

As evident from the table, some personnel in each project area had 
experience in each category of work experience. The range of experience 
is impressive. In all foci personnel indicated experience in RDD&E rang- 
ing from 1 to o^er 16 years. Some personnel had worked on more than 16 
projects and some had no experience on projects at all. Most personnel 
were presently working on one, or at most two or three projects, but a 
few were working on upwards of 16.1 

A few respondents in each project area indicated experience in pro- 
posal writing, experience as project investigator, and experience in 
project management. The range of experience reflected by these items 
again covered the spectrum of response opportunities. 

In terms of experience by project focus, research again appears most 
unique. This consistent difference may be peculair to the projects studied, 
the personnel or. those projects, or the small number of respondents, but 
there are differences that stand out. While research personnel appear to 
have the same broad range of work experience within RDD&E generally, they 
appear to be less experienced in terms of the number of projs^ts worked 
on, the number of proposals worked on, and especially the number of times 
as principal investigator. The number on these projects who are graduate 
assistants partially account for this uniqueness. Evaluation personnel 
stand out in the category Funded Projects How Working On. While personnel 
in research, development, and diffusion tend to work on no more than three 
or four projects at a time, a sizable number of evaluation personnel claim 
to be working on 16 or more projects at one time. These data are influ- 
enced by the experience of staff members in a research and evaluation unit 
within a large school system, and as such^may not be representative of the 
experience of evaluation staff generally. 



1 A consequence of working in the Research and Evaluation Division of a 
large city school system. 

2 

In the evaluation focused projects, four individuals responded that they 
were working on 16 or more funded projects. Because of this "peculiar” 
result the questionnaires from which this data were abstracted were 
reviewed. It was found that all four of the individuals who indicated 
working on so many projects were from evaluation projects currently 
functioning within a large metropolitan school district. In reviewing 
this finding with the team of individuals who visited the site it was dis- 
covered that the projects varied in terms of their size. A number of them 
were quite small, short-term evaluation projects. Many of them, however, 
were fairly large district-wide undertakings. In these cases the indi- 
viduals' time commitment to any given project tended to be quit'j small. 
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In order to provide an added perspective to the data on work exper- 
ience a 9et of questions was asked respondents that focused on the type 
of experience their own work required. Table 6.7 displays these data by 
project focus, 

It will be seen from these data that the range of responses to the 
questions asked was surprisingly broad. While a substantial number of 
personnel suggested that no prior experience was necessary for the per- 
formance of their present job, some personnel suggested that five or more 
years of experience were required. Personnel in research (who have had 
least experience) appeared to value experience less than did other per- 
sonnel, while evaluation personnel seemed generally to value experience 
most highly. Experience al9o 9 eemed to be valued more highly by diffusion 
personnel than by development personnel, especially in relation to the 
performance of administrative and managerial job9. Only one person from 
the entire sample, however, felt that more than 10 years experience was 
required to perform work in any area. 

It is interesting to note that very few individuals felt that any 
experience wa9 required to function in administrative or management roles. 
Thi9 is an interesting result in light of data reported subsequently in 
the volume. Of che 962 outputs of work effort identified in the 20 pro- 
jects studied, 514 were classified as being ’’management related.” Any 
number of hypotheses could account for this apparent discrepancy but in 
light of the increasing awareness of the criticalne99 of the management 
function in large scale RDD&E efforts it is a discrepancy that should be 
investigated . 

Along with responding to work requirement questions, interviewees 
were also asked to indicate the type or level of academic degree they 
considered necessary for the work they were engaged in. Table 6.8 pre- 
sents the data that resulted from responses to thi9 question. The data 
presented in the table are about as expected, given the data previously 
presented on degrees held by RDD&E personnel (9ee Table 6.3, page 85 ) . 
Evaluation personnel emphasize the doctorate proportionately more often 
than personnel in other project areas, though research i9 not far behind. 
Diffusion personnel 9eem to feel the need of the level of academic training 
that accompanies the doctorate least. 

It i9 al9o worthy of note that, with the exception of evaluation, at 
lea9t one person in each project area felt that none of the degrees listed 
was nece99ary for the work they performed. In general it can be concluded 
from Tables 6.8 and 6.3 that RDD&E work, almost without exception, requires 
academic degrees of at lea9t the bachelors level. 



Support Services and Equipment 



To further describe the conditions of work for RDD&E personnel, data 
were gathered with respect to support services and support equipment U9ed. 
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TABLE 6.8 



Level of Academic Training Cited by Interviewees 
As Required by the Position Held 



Required Experience 
Level of Academic 
Training 




Project Focus 




T rif n 1 


Research 

(5) 


Development 

(7) 


1 

Diffusion 

(5) 


1 

Evaluation 

(3) 


lOtcli 

(20) 


No Response 


1 


1 


— 


— 


2 


Doctorate 


6 


8 


4 


9 


27 


Specialist or 
Professional 


2 


5 


2 


3 


12 


Masters 


1 


15 


12 


7 


35 


Bachr 1 ors 


6 


11 


12 


1 


30 


Associate 


— 


— 


— 


— 


— 


Technical License 


— 


— 


— 


— 


— 


None of these ( 


1 


5 


3 


— 


9 


TOTALS 


17 


45 


33 


20 


115 



Table 6.9 displays the variety of support services used by RDD&E personnel, 
by project focus, and Table 6.10 provides the same type of display for 
support equipment . 

The data presented in Table 6.9 make readily apparent the wide range 
of usage of an extensive variety of support services. Each of the 24 ser- 
vices listed is used in some measure by personnel in each of the project 
areas. Virtually the same statement can be made with respect to the 12 
types of support equipment listed in Table 6.10. 

Far outweighing all other categories of support service is that of 
typing. The relatively high frequency of printing, additional secretarial 
service, and computer analysis services cited irrespective of project focus, 
is also noteworthy. Little wonder the oft heard claims that "An organiza- 
tion runs because of its secretaries.", and "We’re drowning in paper." 

The support services used by at least 50% of project staff, without 
respect to project focus, are: Printing; Secretarial Services other than 
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TABLE 6.9 



Frequency of Support Service Use 



Support Services Provided 




Project Focus 








by other Persons 
or Agencies 


Res . 


Dev. 


Diff. 


Eval. 


Total 

115 


Total No. of Respondents 


17 


45 


33 


20 




i. 


Typing 


14* 


41* 


29* 


17* 


101* 


2. 


Printing 


10* 


32* 


21* 


16* 


79* 


3. 


Secretarial service, 
other than typing 


12* 


30* 


21* 


10* 


73* 


4. 


Computer analysis services 


13* 


21* 


23* 


15* 


72* 


5. 


Purchase cf supplies & equip 


11* 


19 


23* 


12* 


65* 


6. 


Library holdings 


9* 


24* 


19* 


13* 


65* 


7. 


Budgetary & other fiscal 
accounting 


11* 


19 


19* 


10* 


59* 


8. 


Other reproduction services 


10* 


24* 


13 


11* 


58* 


9. 


Statis tical consul tation 


10* 


13 


18* 


16* 


57 


10. 


Computer program writing 


13* 


12 


15 


15* 


55 


11. 


Subjects for experimentation 
or try-out of procedures 


7 


24* 


15 


9 


55 


12. 


Audio-visual aids and devices 


5 


24* 


19 


6 


54 


13. 


Subscriptions to techn. & prof, 
journals /periodicals 


9* 


10 


21* 


12* 


52 


14. 


Travel arrangements 


11* 


14 


19* 


6 


50 


15. 


Scoring of test items 


6 


17 


19* 


7 


49 


16. 


Editing 


6 


17 


18* 


7 


48 


17. 


Art work and illustrations 


6 


15 


17* 


9 


47 


18. 


Requests for documents or pub- 
lications not locally avail. 


7 


10 


18* 


9 


44 


19. 


Photography 


3 


10 


14* 


3 


30 


20. 


Technical writing 


4 


6 


13 


5 


28 


21. 


Television facilities & equip* 


3 


7 


8 


5 


23 


22. 


Other services 


3 


7 


5 


4 


19 


23. 


Drafting 


3 


5 


6 


3 


17 


24. 


Equipment construction 


3 


2 


8 


2 


15 



* Represents use by at least half of respondents. 
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typing; Typing; and Library holdings. Support services utilized by at 
least 50% of project staff within three project foci would increase the 
list to include: Other reproduction services; Purchase of supplies and 

equipment; Subscriptions to technical and professional journals/periodicals; 
Computer analysis services; Statistical consultation; and Budgetary and 
other fiscal accounting. 

The data presented in Table 6,10 are equally impressive in demon- 
strating the heavy reliance of RDD&E personnel on equipment. With one 
exception, all items listed were in use in all project foci. The two 
most frequently used pieces of equipment were desk calculators and 
dictating equipment. While desk calculator usage appears to be fairly 
well distributed across research, development, diffusion, and evaluation 
projects, dictating equipment appears to be used most often in develop- 
ment projects. In general it would appear as though research and evalua- 
tion personnel utilize equipment which is oriented toward the reduction 
of data, either by hand or computer, more often than development and diffu- 
sion personnel. On the other hand, development and diffusion personal 
apparently make use of photographic and video tape equipment more o n 
than staff on research and evaluation projects. The equipment and support 
needed on a project is, of course, determined to a great extent by the 
nature of that project. This fact must be considered in interpreting 
the data presented in Table 6,9 and 6.10. 



Supplementary Observations 



A number of additional observations about RDD&E personnel are note- 
worthy. First, it was not unusual for the staff of a project to vary 
markedly in its composition over time. In one of the projects studied it 
was noted, for example, that a complete staffing change was made after the 
first year of operation. This resulted from the project having been 
initiated by senior staff members of the sponsoring organization, and that 
as soon as the planning, designing, and start up phase of the project had 
ended a production oriented staff took their place. 

A second observation, and one that squares with data reported in the 
chapter, was that a sizable number of staff members associated with pro- 
jects held multiple project assignments. In most of these instances the 
percentage of time dedicated to one project was minimal, and the role they 
tended to play was one of "reviewer and critiquer" of daily work produc- 
tion and/or "planner and director" of long range work activities. Such 
persons also tended to spend considerable time and energy in contact with 
other individuals and agencies engaged in similar work. It was not at all 
unusual for persons of this kind to be unable to respond to detailed ques- 
tions about project operations. 

Another phenomenon observed frequently in the projects visited was the 
subcontracting of major portions of work activity to other individuals and 



TABLE 6.10 



Frequency of Support Equipment Use 



Support Equipment Immediately 


Project 


Focus 








Available and Used by 
Project Respondents 


Res. 


Dev. 


Mff. 


Eval . 


Total 

115 


Total No. of Respondents 


17 


45 ! 


33 


20 


i. 


Desk calculators 


11* 


18 


11 


16* 


56 


2. 


Dictating equipment 


7 


25* 


12 


9 


53 


3. 


Key-punch machine 


14* 


9 


12 


11* 


46 


4. 


Other equipment 


9* 


19 


13 


3 


44 


5. 


Data card sorter 


12* 


8 


5 


10* 


35 


6. 


Video tape 


3 


15 


9 


6 


33 


7. 


On-site computer 


9* 


7 


3 


12* 


31 


8. 


Readers for microfiche 
or microfilm 


4 


5 


13 


7 


29 


9. 


Photographic equip. 


2 


11 


10 


5 


28 


10. 


Remote computer terminal 


9* 


7 


3 


2 


21 


11. 


Television camera 


2 


4 


7 


4 


17 


12. 


Desk-top computer 


1 


0 


5 


9 


15 



* Represents use by at least half of respondents. 



agencies. These subcontracts generally took the form of formal agree- 
ments, but they also included a range of advisory and consultant arrange- 
ments. The variation in the background experiences of the advisors and 
consultants encountered in this connection was remarkable. 

The staffing patterns encountered in projects also showed a great 
deal of variability. Within the 20 projects studied staffing patterns 
varied from two men "sharing" project responsibilities to a complex, and 
highly structured "corporate" staffing arrangement. It also varied with 
the size and setting of a project, the role played by subcontractors and 
consultants, and the particular outputs for which the project was respon- 
sible. As a consequence, the overwhelming impression gained in regard to 
staff structure was one of variability. In almost every project, how- 
ever, care was taken to point out that the organizational structure adopted 
was "functional" rather than "formal" in nature. While that distinction 
was never made perfectly clear, a functional organization apparently calls 
for an alignment of staff by work activities rather than job titles or 
organizational boxes. On the basis of that which has been observed, however, 
it is doubtful that constructs such as pyramidal and corporate structures 
can accommodate the variability that exists in staff arrangements in the 
domain of educational RDD&E. 
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Chapter 7 



OUTLINE MAP B: OUTPUTS OF EDUCATIONAL RDD&E 



This is the second of the three chapters that organize the data col- 
lected in the Oregon Studies around the focus of the projects studied. It 
is the first to report the data that are central to the studies, and as 
such, serves as an anchor for the data reported in subsequent chapters. The 
aim of the chapter is to describe the outputs of work effort identified with- 
in the 20 projects studied. 

As used in the Oregon Studies, an output is defined as "an identifi- 
able outcome of targeted work activity that contributes to the realization 
of project goals.' 1 So conceived outputs vary considerably. They can take 
the form of a contracted-for outcome of a project, for example a final re- 
port of research done, a mediated instructional system, a set of trained 
diffusion agents, or they can take the form of any of the hundreds of "out- 
puts" required to produce a contractual outcome, for example data collection 
instruments, a computer program, a literature review, The various instruc- 
tional elements that make up a mediated instructional system. They can also 
take the form of field trial results of an instructional system or the PERT 
diagrams used by persons responsible for producing such a system to insure 
that all elements within it are completed on a given time line. Outputs can 
assume the form of "hard products," such as those just described, or they can 
assume the form of events or conditions. Events, such as conferences, staff 
meetings, and the selection of personnel, represent the outputs of work effort 
just as surel> as hard products. So also do conditions such as competently 
trained diffusion agents, parents cooperating in curriculum planning, and 
good staff morale. 

Outputs, then, may be big or little, hard or soft, and essentially in- 
finite in their substantive focus# The defining feature of outputs is that 
of being an identifiable outcome of targeted work activity that can be shown 
to contribute to the realization of the goals of a particular project. The 
decision rules governing the iaentif ication of outputs are described in detail 
in Volume V of the series of volumes reporting the Oregon Studies. 

It will be recalled from Chapter 4 that a 2-step process was involved 
in identifying the outputs of a project: (a.) establish a list of all out- 

puts associated with the project (an output index); and (Jb) order those 
outputs according to their interdependencies (an output map). A wide variety 
of sources were used in developing the output index. Any and all documents 
describing the work of a project were consulted, including the project pro- 
posal and work management charts. Project directors and other persons in 
management positions were approached specifically for a listing of project 
outputs. Finally, all project staff were asked to review, revise and add to 
tentative listings of project outputs. 

Once identified, outputs were "mapped" in terms of their interdepen- 
dencies. Much the same process was followed here as in output identification. 



Documents were referred to; project directora/managers were interviewed; 
and staff were asked to review and revise approximations to the map. 

The process of establishing an output index and map for a particular 
project was begun at the time the director of a project was first con- 
tacted about participation in the Oregon Studies. It continued throughout 
the preliminary project review process and resulted in a data collection 
team arriving at a given site with a firm approximation to an output index 
and map for that site. The first day on the site was spent refining and 
confirming the two documents. Subsequent modifications were free to occur 
as additional information was gained about a project, but these tended to 
be few. Only outpats identified by project staff as significant to the 
operation and well being of a project were included in an output index and 
map. Each case pnfile is to a large extent designed around these two 
documents . 

Nine hundred sixty-two outputs were identified in the 20 projects 
studied. The outputs identified for any single project varied from 14 to 
205. The number of outputs identified in each project studied are listed 
in Table 1. 



TABLE 7.1 

The Number of Outputs Identified in the Projects Studied 



Project 

Number 3 


Outputs 

Identified 


Project 

Number 


Outputs 

Identified 


1 


40 


11 


22 


2 


20 


12 


18 


3 


14 


13 


67 


4 


28 


14 


44 


5 


17 


15 


70 


6 


67 


16 


57 


7 


48 


17 


57 


8 


15 


18 


47 


9 


30 


19 


205 


10 


65 


20 


32 



As listed sequentially in Volume IV of the series of volumes reporting 
the Oregon Studies. 



The Classification of Outputs 

The task of giving order to the outputs identified was a major one. 
The decision was made early in the Oregon Studies not to impose precon- 
ceived category sets upon the data collection process, but rather to let 
category sets emerge from the data collected in the field. Procedurally 
this required that outputs be listed sequentially as they were identified. 



within and across projects, and that commonalities be found between them. 
Three classification systems were developed for purposes of identifying 
commonalities between outputs. These were labeled , respectively, as 
the primary system, the clust er system, and the dimensional system. 



Prim ary Categories of Output s 

In keeping with the commitment to have the category sets that describe 
outputs emerge from the language of persons in the field, the initial 
classification given outputs involved labels that had "intrinsic" meaning, 
and that were "close to the source." Operationally, this meant that cate- 
gory labels describing output commonalities involved a low level of 
abstraction and corresponded closely to the descriptions of outputs pro- 
vided by persons in the field. Categories descriptive of output common- 
alities at this first level of abstraction were labeled primary output 
categories. Three hundred twenty-six primary categories were established 
to code the 962 outputs identified. The full listing of primary cate- 
gories, and the number of outputs classified within them, are provided in 
Appendix 1. 



Cluster Categories of Outp uts 

As the number of primary output categories grew it became increasingly 
difficult to apply them V7ith efficiency and reliability to the classification 
of outputs. As a consequence, in order to group the primary categories into 
manageable "clusters" for purposes of output classification, and in order 
to better comprehend the nature of the variability within the primary cate- 
gories, a second order or cluster set of categories was developed for the 
primary categories. The guiding principle in developing the cluster cate- 
gories for outputs was also one of maintaining the intrinsic meaning of 
output labels. Beyond that, however, and the traditional requirement of 
discreteness, clarity, and operational definition, the cluster categories 
were free to emerge as dictated by the content of the primary category set. 

Fifty-one cluster categories were established to code the 326 primary 
categories. Appendix 2 contains a complete description of output: cluster 
categories and the frequencies with which they were used to classify 
outputs appearing in the primary category sets they incorporate. The 
primary categories incorporated within each cluster category are listed 
in Appendix 3. 

In distinguishing between primary and cluster output categories, and 
understanding the relationships between them, it is important to realize 
that primary categories are free to occur within any cluster category. 

Table 7.2 illustrates this relationship between the two category sets by 
showing two different cluster categories that share a common primary output 
category. While such overlapping is free to occur, in practice it is a 
relatively infrequent phenomenon. 

Two additional items need to be taken into account in understanding the 
relationship between primary and cluster output categories. First, both are 
treated as open sets. As additional outputs are identified In subsequent 
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studies, additional primary categories, if needed, may be developed to 
accommodate them. The same holds with respect to cluster categories. If 
additional classes of primary * categories emerge that cannot be handled in 
the existing cluster set, new cluster categories may be developed. 

Second, the total number of outputs classified as falling within a cluster 
category is not necessarily a reflection of the number of primary categories 
that make up a cluster category. The data contained in Table 7.2 demon- 
strate this point. In the table, 122 outputs and 20 primary categories of 
outputs are shown as falling within cluster category 01. In cluster cate- 
gory 05, however, which contains a larger number of primary output categories 
(26) only 62 outputs are identified as falling within the cluster category. 
While such an arrangement is not uncommon in hierarchically ordered category 
sets, the reader needs to be attuned to the fact that the frequency with 
which a cluster category is used (the number of outputs coded within it) is 
not directly related to the number of primary categories it subsumes. Again, 
the reader is directed to Appendix 3 for a complete listing of the primary 
output categories that make up the cluster categorieSj and the frequency 
distribution of both by project focus. 



Categories Describing the Dimensions of Outputs 

At the same time that primary and cluster category sets were being 
developed for the coding of outputs, an effort was made to identify the various 
dimensions on which outputs varied. The first dimension to emerge with any 
degree of clarity was that which has been labeled the s tructure of outputs. 
Early in the Oregon Studies the conception held of outputs was product 
oriented, that is, outputs were thought of only as being tangible or "hard" 
outcomes of work effort that were concrete in form and transportable at some 
point in time. As the studies progressed, however, and the data on outputs 
of projects accumulated, it became clear that RDD&E projects often sought 
outputs of work effort that were not tangible or transportable. Ultimately 
it was recognized that events and conditions were also outputs of work effort. 
This recognition led to the present structural classification of outputs 
as being either products, events, or conditions. 

Operationally, all outputs were identified as falling into one of these 
three categories. As such, the structure of outputs served as the primary 
organizer of outputs within the primary and cluster sets, that is, each 
output identified received a structural classification as well as a primary 
and cluster classification. The categories used in classifying output struc- 
ture were defined as follows: 

Product - An outcome of work effort that is tangible or concrete 
in form, and transportable at a given point in time, 
e.g., a single concept film, a budget, a book. 



As they presently stand the cluster categories have received no intensive, 
"after the fact" analysis as to discreteness, overlap, equivalence in level 
of generality, etc. An extremely useful follow-on activity to the Oregon 
Studies would be such an analysis. 
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Event - An outcome of work effort that results in the occui rence 

of an observable transaction or set of behaviors, e.g., 
a seminar, a staff meeting, a field test. 

Condition - An outcome of work effort that creates a desired circum- 
stance expected to endure over the life of j. project, 
or as a result of it, e.g., parental involvement in 
planning school curricula, good staff morale* . 

Three additional dimensions of project outputs were identified during 
the course of the Oregon Studies. These were the dimensions of function , 
character , and level . The f anr.tional dimension of outputs refers tc the 
use made of an output within r.he context of a particular project; and the 
level dimension refers to the hierarchical standing of an output within the 
context of a particular project. As used in the Oregon Studies, these 
output dimensions were defined as follows: 

Output Function - The attributes of an output that mark it as 

serving a policy, management, or production 
f unc tion. 

Output Character - The attributes of an output that mark it as an 

instance of knowledge, technology, implementa- 
tion, or information. 

Output Leve,i - The attributes of an output that identifies its 

relationship to project goals as focal, component, 
or facilitating. 

The data presented in the present chapter deal only with the structural 
attributes of outputs. Data describing the function, character, and level 
of outputs are presented in Chapter 9. 



Primary Categories of Outputs 

Table 7.3 displays the number of primary output categories used to 
describe the products, events, and conditions identified within the 20 
projects studied. The table needs to be read as follows: a total of 326 

categories were developed to classify the 962 outputs identified. Two 
hundred forty-five of the 326 were used to classify products, 43 to classify 
events, and 38 to classify conditions. Of the 245 product categories 40 
were used to classify products identified in the five research projects 
studied, 125 were used to classify products identified in the seven develop- 
ment projects studied, etc. Clearly, the most noticeable feature of the 
table is the p re dominance of product categories o^er event and condition 
categories* Tha large number of product categories is a reflection of two 
factors: (*&) the remarkable number of "hard outputs 11 that emerge from pro- 

jects; and (bi) the fact that the event and condition codes were not used 
until midway through the Studies. Once these category sets emerged, all 



TABLE 7.3 



The Number of Primary Output Categories Used to 
Describe Products, Events, and Conditions 



Output 

Structure 


No . of 
Categories 
Derived 


Project Focus 


Res . 
(5) 


Dev. 

(7) 


Diff . 
(5) 


Eval . 
(3) 


Products 


245 


40 


125 


139 


52 


Events 


43 


18 


15 


27 


12 


Conditions 


38 


11 


14 


24 


10 


TOTALS 


326 


69 


154 


190 


74 



previously coded outputs were recoded to determine if outputs that had pre- 
viously been forced to fit the classification of product were better coded 
with the new categories. As expected, some did, but undoubtedly many out- 
puts that would have fit the definition of events and conditions were missed 
in the earlier data collection procedures. So, even with recoding the 
originally collected data, and applying the codes from midway in the studies 
on, the total pool of events and conditions around which to build primary 
category sets remained relatively small. This is reflected in the rela- 
tively small number of categories in Table 7.3 for use in coding events and 
conditions. This imbalance, and the reasons for it, needs to be kept in 
mind when considering the data presentations and discussions that appear 
in the paragraphs that follow. 



Produc ts 



The 245 primary categories used to describe products are displayed in 
Appendix 1. The total number of products coded within the 245 categories 
sums to 759. The number of products coded within any single category ranges 
from one to 24, with 113 of the 245 categories containing only one output 
entry. Table 7.4 displays, by project focus, the five categories used most 
frequently to classify products. 

Three aspects of the data presented in the table are noteworthy: 

(a_) the small number of products coded within any one category; (b) the 
fact that all five categories were used across all four project foci; and 
(c) the uneven use patterns of some categories across FDD&E, for example, 
Evaluation Report and Descriptive Data Tests, and the relatively even 
use of others, for example, Project Time Line/Schedule and Terminal 
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TABLE 7.4 



The Five Categories Used Most Frequently 
to Classify Products 



Code 


Primary Categories 


Project Focus 


Totals 


NO 


of Outputs 


Res . 


Dev . 


Diff . 


Eval . 


(20) 






(5) 


(7) 


(5) 


P) 




0158 


Evaluation Report 


1 


1 


12 


10 


24 


0013 


Project Time Line/Schedule 


4 


6 


7 


3 


20 


0187 


Descriptive Data Tests, Subject 


14 


1 


3 


1 


19 


0006 


Terminal Report 


6 


3 


4 


3 


16 


0082 


Questionnaire 


2 


5 


6 


2 


15 


Total number of products coded 


27 


16 


32 


19 


94 


within the five categories 


Total number of products identified 


83 282 


295 


99 


759 


Percentage of products coded with 
the five categories 


33% 


06% 


11% 


19% 


12% 



Report. The significance of Table 7.4, however, lies not so much in the 
data it presents as in the data it does not present. The relatively few 
products coded within the five categories listed, and the necessarily 
infrequent use of the 238 categories of products not appearing in Table 
7.4, gives evidence of the wide and varied range of products produced 
within the various project foci. Because of variability found within the 
product data, and the low frequencies involved, the reader is encouraged 
to inspect Appendix 1 closely. 



Events and Conditions 



The reader will see from an inspection of the events and conditions 
categories listed in Appendix 1 that essentially the same pattern exists 
theiv' that exists for products. Like products, they have extremely 
small frequencies and vary widely across project foci. Of the 43 
primary categories used to describe events, only three contain out- 
puts that crosscut all project fcci* Primary event category 0241 
Selecting Personnel occurred more frequently than any other (16 times) 
and was cited in all focus areas* Diffusion projects accounted for 

nine of those citations, however. Primary event category 0176 Training 

Workshop Conference also occurred in all project foci. It was cited 
a total of nine times. Output 0012 Assigning Work was the lone 
remaining category of event to be cited across all project foci, and it 
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was cited only five times. Twenty of the A3 primary categories of events 
were used to classify a single event. Of all events identified 23 were 
found within research projects, 19 within development projects, 67 in 
diffusion projects and 21 in evaluation projects. 

The most frequently cited category of conditions occurs only seven 
times. The category Adequate Staff is one of only two conditions to 
he represented in all four project foci. The other, Staff Welfare, is 
cited only five times. Of the remaining 36 categories within the set, 
none were used more than four times to classify outputs and over half 
were used only once. 



As in the case of the primary categories descriptive of products 
the reader is encouraged to inspect Appendix 1 from the point of view of 
the nature and range of the events and conditions categories cited there, 
as well as their distribution by project focus. In light of the project 
sample, however, and the late appearance of the event and condition codes, 
care must be taken in interpreting these data. 



Cluster Categories of Outputs 

Table 7.5 summarizes the number of cluster categories used to 
describe the products, events, and conditions identified within the 20 
projects studied. The table is to be read in the same way as Table 7.3. 

TABLE 7.5 

The Number of Output Cluster Categories Used to 
Describe Products, Events, and Conditions 



Output 

Structure 


No. of Cluster 
Categories 
Derived 


Project Focus 


Res. 

(5) 


Dev . 

W) 


Diff . 

(5) 


Eval . 
(3) 


Products 


20 


11 


16 


16 


10 


Events 


19 


12 


9 


17 


9 


Conditions 


12 


9 


6 


9 


8 


TOTALS 


51 


32 


31 


42 


27 



As will be seen from the table, the wide disparity in number of 
primary categories used to code products, events, and conditions is not 
maintained in the cluster categories. Twenty cluster categories were 
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needed to rode the 74b primary categories of products; 19 cluster cate- 
gories were needed to code the 43 primary categories of events; and 12 
cluster categories were needed to code the 38 primary categories of con- 
ditions. Obviously, the number of primary categories subsumed under the 
product clusters exceeds the number under the event and condition clusters, 
but in absolute terms they are reasonably comparable in number. Illus- 
trations of the cluster categories used in describing products, events, 
and conditions 3re provided in Table 7.6. 





TABLE 7.6 


Examples of Output Cluster Categories that Describe 
Products, Events, and Conditions 


Output 

Structure 


Examples of Cluster Categories 


Product 


Guides /Manuals ; Critiques; Publications 


Event 


Training; Staff Meetings; Field Trials/ 
Experiments 


Condition 


Accountability; Informed Public; Trained 
Staff 



Products 



The twenty cluster categories used to describe products, and the 
frequency with whicn they were used across project foci, are presented 
in Table 7.7. Three features dominate the data presented in the table; 

(ci) the fact that approximately half of the cluster categories were used 
frequently and half were used infrequently; (Id) the fact that of the 
frequently used categories only four (products 01, 27, 02, and 03) were 
used consistently across all four project foci; and (c) the fact that a 
number of the categories that had reasonably frequent use appeared within 
one, or at most two, project foci (products 04, 08, 03, 10, 07, 14, 12, 13, 
and 11). As such, there is a lot of variability in the cluster category 
data. Only products that share the characteristics of Reports/Contracts, 
Data, Data Instruments/Techniques, and Work Management Systems were 
shared by all project foci with any degree of frequency. Work Specifica- 
tions/Procedures and Plans/Designs/Theories and Models were the most 
frequently cited of the categories included in the development focus. 

Also, it was no surprise to find that diffusion projects produced by far 
the greatest number of Guides/Manuals and Publications. By and large, 
the differences observed between RDD&E projects tend to square with 
differences expected . 



TABLE 7.7 



The Cluster Categories Used to Describe Products, and Their 
Distribution by Project Focus 



Set DE 
Cod ing 


Output Cluster Category 
(Output Structure: 
Code Set A-l) 


Project Focus 


Output 

Totals 

(20) 


Products 


Res . 
(5) 


Dev . 
,< 7 > 


Diff . 
(5) 


Eval . 
(3) 


01 


Reports /Contracts 


21 


25 


47 


29 


122 


04 


Work Specifications/Procedures 


5 


45 


21 


7 


78 


27 


Data (including printouts) 


10 


26 


25 


14 


75 


02 


Data Instruments/Techniques 


20 


22 


18 


14 


74 


08 


Plans/ Designs /Theories/Models 


8 


44 


14 


7 


73 


05 


Work Management Systems 


7 


20 


23 


12 


62 


03 


Guides/Manuals 


3 


18 


29 


6 


56 


10 


Publications 


4 


6 


36 


6 


52 


07 


Resource Lists 


2 


31 


11 


0 


44 


14 


Instructnl Units/Use w/Staff 


2 


4 


30 


3 


39 


12 


Instructnl Units/ In-School 


0 


26 


3 


0 


29 


13 


Instructnl Units/Extra-School 


0 


0 


18 


0 


18 


11 


Promotional Materials 


0 


1 


16 


0 


17 


28 


Taxonomies 


0 


5 


0 


0 


5 


29 


Recommendations 


0 


4 


0 


0 


4 


36 


Curricular Objectives 


0 


3 


0 


0 


3 


37 


Critiques 


0 


2 


1 


0 


3 


06 


Data Management Systems 


1 


0 


1 


0 


2 


09 


Equipment/ Tools /Facilities 


0 


0 


2 


0 


2 


42 


Prediction Variables 


0 


0 


0 


1 


1 


TOTALS 


83 


282 


295 


99 


759 



Events 



The nineteen cluster categories used to describe events, and the 
frequency with which they were used across project foci, are presented 
in Table 7.8. As will be seen in the table, the use put tern of these 
categories approximates the use pattern for the categories that describe 
products, although their frequency of use is much smaller. This coincides, 
of course, with the smaller number of events identified in the projects 
studied. Nevertheless, a number of the categories were called upon to 
describe events that occurred in all project foci (categories 32, 33) or 
at least three of the four foci (categories 17, 40, 31, 38). Thus, 



TABLE 7.8 



The Clutter Categories Used to Describe Events, and Their 
Distribution by Project Focus 



Set; DE 
Coding 


Output Cluster Category 
(Ou tpu t Structure: 
Code Set A-2) 


Project Focus 


Output 
To t al s 


Events 


Res . 

151 


Dev . 
(7) 


Diff . 
(5) 


Eval . 
(3) 


(20) 


32 


Recruiting/Placing Staff 


3 


3 


10 


5 


21 


33 


Collecting/Analyzing Data 


4 


2 


5 


5 


16 


52 


Adapting Materials 


0 


0 


12 


0 


12 


43 


Establishing Opns Parameters 


0 


0 


9 


2 


11 


17 


Workshops/ Institutes 


3 


4 


3 


0 


10 


40 


Acquiring Supplier /Facilities 


2 


3 


5 


0 


10 


31 


Field Trials /Experiments 


1 


3 


3 


0 


7 


38 


Training 


0 


1 


2 


4 


7 


51 


Acquiring Funds 


0 


0 


6 


0 


6 


48 


Maintaining Accountability 


2 


0 


.3 


0 


5 


18 


Presentations 


1 


0 


2 


1 


4 


19 


Benchmark Decisions 


2 


0 


1 


1 


4 


50 


Disseminating Info/Outputs 


1 


0 


2 


1 


4 


30 


Committee/Consultant Meetings 


0 


1 


1 


1 


3 


15 


Staff Meetings 


1 


0 


0 


1 


2 


16 


Conferences/ Seminars 


0 


1 


1 


0 


2 


39 


Updating of Machine Systems 


0 


1 


1 


0 


2 


49 


Establishing Field Sites 


1 


0 


1 


0 


2 


47 


Coordinating Efforts 


2 


0 


0 


0 


2 


TOTALS 


23 


19 


67 


21 


130 



Recruiting/Placing Staff, Training Staff, Collecting/Analyzing Data, 
holding Workshops/institutes, Acquiring Supplies/Facilities, and carrying 
out Field Tr ials/Fxper iments , are events that constitute relatively common 
occurrences within the domain of RDD&E. Adapting Materials, Establishing 
the Parameters of Operations, and oddly enough. Acquiring Funds, were 
identified as critical events only within diffusion projects. Apart from 
these differences, the general sense gained from the data presented in 
Table 7.8 is that had larger numbers of events been identified in the 
20 projects studied the use pattern of the event cluster categories would 
closely follow that of the product cluster categories. 



Conditions 



Only 12 cluster categories were required to code the primary cate- 
gories of conditions. These categories, and the frequency with which they 
were used in classifying conditions by project focus, are presented in 
Table 7.9. 

Approximately the same use pattern will be seen for these categories 
as was seen for the cluster categories descriptive of products and events. 
A numbev of the categories were used across all project foci, a number 
across three of the four foci, and a number in only one. Also, there 
were no great surprises a9 to the nature of the categories used within 
particular foci. For example, a proper Working Environment/Atmosphere 
and Coonerative Relationships are conditions that most if not all pro- 
jects strive to achieve. The reader is sufficiently familiar with the 
nature of the data appearing in Table 7.9 that further comment about it 
is necessary. 



TABLE 7.9 

The Cluster Categories Used to Describe Conditions, and Their 
Distribution by Project Focus 



Set DE 
Coding 


Output Cluster Category 
(Output Structure: 
Code Set A-3) 


Project Focus 


Output 

Totals 

(20) 


Conditions 


Res . 
(5) 


Dev. 

(7) 


Diff. 

(5) 


Eval. 
_ (3) _ 


21 


Working Environment/Atmosphere 


3 


2 


11 


3 


19 


20 


Cooperative Relationships 


2 


5 


6 


1 


14 


34 


Coordinated Efforts 


1 


0 


8 


1 


10 


22 


Quality Assurance 


1 


3 


3 


0 


7 


23 


Accountability 


2 


2 


n 


1 


6 


25 


Resource Accessibility 


1 


0 


2 


1 


4 


41 


Effective Communication 


1 


1 


2 


0 


4 


44 


Provision for Services 


0 


0 


2 


2 


4 


45 


Informed Public 


1 


0 


0 


1 


2 


26 


Available Field Sites 


1 


0 


0 


0 


1 


35 


Acceptance/Adoption of Outputs 


0 


0 


1 


0 


1 


46 


Trained Staff 


0 


1 


0 


0 


1 


TOTALS 


13 


14 


36 


10 


73 
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The primary contribution of this chapter lies in making explicit the 
range, form, and substance of those things to which the data in subsequent 
chapters are linked. As such it provides a view of the broad range of 
concerns on which work activity is actually targeted in calculated fashion. 

J From this view, the comprehensiveness of the work requirement data con- 

tained in the next chapter can be anticipated. 

i 
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Chapter 8 



OUTLINE MAP C: WORK REQUIREMENTS WITHIN EDUCATIONAL RDD&E 



This is the third of the three chapters that organize the data 
collected in the Oregon Studies by project focus, that is by research, 
development, diffusion, and evaluation. Chapter six provided a summary 
of the data collected on the people associated with the 20 projects 
studied; Chapter 7 provided a summary of the outputs of work effort iden- 
tified within the 20 projects studied; and the present chapter provides 
a summary of the work requirements data from the 20 projects studied, 

Ti terms of the information needed by the designers of personnel develop- 
ment programs, therefore, or by persons entering or working within the 
field, the data presented in the present chapter may prove to be the most 
interesting of the three. 

The work requirement data to be reported in the chapter pertains to 
the elements of work associated with the production of the outputs ana- 
lyzed in the 20 projects studied. It wiJ 1 be recalled from Chapter 4 
that three major classes of work data were collected for each output ana- 
lyzed: (ji) the standards held for an output; (J)) the tasks involved in 

producing an output to its specified standards; and ( c ) the enablers 
(knowledges, skills, and^sensitivities) relied upon or utilized in the 
production of an output. It will also be recalled that these data were 
collected in the form of descriptive statements from orojact staff about 
the nature of the work required to produce the particular outputs for which 
they were responsible. These statements were collected through on-site 
interviews, and subsequently reduced to category codes. Technical defi- 
nitions of standards, tasks, and enablers, as well as an overview of the 
codes used in their classification are provided as each of these classes 
of data is encountered in the pages that follow. 

Two sensitivities need to be provided about the data reported in the 
present chapter. First, work r quirement data were obtained for only a 
sample of the total pool of outputs identified in the 20 projects studied. 
Of 962 outputs identified, only 298 were analyzed for purposes of identi- 
fying work requirements. Since thxi represents less than a third (31%) 
of the total number of outputs identified within uhe 20 projects studied, 
the adequacy of the sample of outputs analyzed is critical to the adequacy 
of the work requirement data presented. Because of the criticalness of 



Actually, standards describe an attribute of outputs, but because they 
influence so directly the work required to produce outputs they have been 
clustered for reporting purposes as an aspect of work requirement data, 

2 

Data obtained by means of questionnaire on the work activities of per- 
sonnel in the 20 projects studied are reported in Chapter 12, 
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this issue, attention is directed to the relationship between outputs 
identified and analyzed before the work requirement data are presented. 

The second sensitivity that needs to be provided about the work 
requirement data as they are reported in the present chapter is that they 
are presented as aggregate data (as if they were independent of the outputs 
from which they take their meaning). They are reported as standards, tasks, 
knowledges, etc., that are used within the domain of educational RDD&E 
without reference to the outputs to which they are linked. While this pro- 
cedure has the utility of allowing the work requirements that exist within 
the domain to be "mapped 11 in outline form, it takes those data out of their 
appropriate context, and thereby affords the risk of misinterpretation and 
loss of meaning. The use of the work requirement data in this way is 
countered in Chapters 10 and 11 where they are presented in terms of their 
appropriate linkage to outputs. 



The Relationship Between Outputs Analyzed 
and Outputs Identified 



As indicated above, the data to be reported in the pages that follow 
were derived from an analysis of the work requirements associated with 
the production of 298 of the 962 outputs identified in the 20 projects 
studied. Tables 8,1, 8,2, and 8.3 provide a summary of the relationship 
between identified and analyzed outputs. The tables are organized according 
to the structural properties of outputs, that is, by products, events, 
and conditions. The data reported within them are in the form of cluster 
categories (see Chapter 7, pp, 107 to 112 ) . Accordingly, Table 8,1 provides 
a summary of the cluster categories of products identified and analyzed; 

Table 8,2 provides a summary of the cluster categories of events identified 
and analyzed; and Table 8,3 provides a summary of the cluster categories of 
conditions identified and analyzed. 

Three features of the data presented in these tables are noteworthy: 

GO the large number of products that have been analyzed as compared to 
events and conditions; (b) the smaller proportion of identified products 
analyzed as compared to events and conditions; and (c_) the reasonably ex- 
haustive coverage of the categories of outputs identified by those analyzed. 

Each of these features needs to be dealt with separately. The imbal- 
ance in the number of products, events, and conditions analyzed reflects 
the imbalance in the number identified. As pointed out in Chapter 7, two 
factors accounted for this imbalance; the large number of hard products 
produced within projects, and the fact that in the earlier stages of the 
Oregon Studies the concept of events and conditions as project outputs did 
not exist — at least not formally. As a consequence, the majority of outputs 
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identified and thus analyzed in the first 10 projects studied took trie 
form of products. ^ 



TABLE 8.1 



The Distribution of Outputs Identified and Analyzed 
By Product Clusters 



Set 

DE 

Coding 


Output Cluster: 
Products 


Frequency 


of Outputs 




Identif ied 


Analyzed 


01 


- — ■ - • — — — ‘ 

Reports/Contracts 


- 

122 


26 




04 


Work Specifications/Procedures 


78 


21 




27 


Data (including printouts) 


75 


13 




02 


Data Instruments/ Techniques 


74 


20 




08 


P Ians /Designs /Theories /Mod els 


73 


26 




05 


Work Management Systems 


62 


19 




03 


Gu id e s / Manu a 1 s 


56 


9 




10 


Publications 


52 


7 




07 


Resource Lists 


44 


13 




14 


Instructnl Units/Use w/Staff 


39 


5 




12 


Instructnl Units/In-School 


29 


13 




13 


Instructnl Units/Extra-School 


18 


3 




11 


Promotional Materials 


17 


1 




28 


Taxonomies 


5 


2 




29 


Recommendations 


4 


— 




36 


Curricular Objectives 


3 


— 




• 37 


Critiques 


3 


1 




06 


Data Management Systems 


2 


2 




09 


Equipment/Tools/Fac LI i ties 


2 


— 




42 


Prediction Variables 


1 


1 




TOTALS 


759 


182 





A number of the "products" identified in the early data collection 
efforts were related to what was called "management responsibilities." 
Subsequent analyses showed that such "products" were often equivalent to 
what is now labeled an event or condition. The data collected in the 
early stages of the Studies have been recoded with the presently exist- 
ing category sets, and, as a consequence, many of the coding errors intro- 
duced in the early data as a result of inadequate categories have been 
corrected The fact remains, however, that the total pool of outputs 
analyzed has been biased by the orientation to outputs held In the early 
stages of data collection. 
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TABLE 8.2 



The Distribution of Outputs Identified and Analyzed 
By Event Clusters 



Set 

DE 

Coding 


Output Cluster: 
Events 


Frequency 


of Outputs 


Identified 


Analyzed 


32 


Recruiting/Placing Staff 


21 


14 


33 


Collecting/Analyzing Data 


16 


4 


52 


Adapting Materials 


12 


4 


43 


Establishing Opns Parameters 


11 


11 


40 


Acquiring Supplies/Facilities 


10 


3 


17 


Workshops / Ins t i tu tes 


10 


3 


31 


Field Trials/Experiments 


7 


2 


38 


Training 


7 


3 


51 


Acquiring Funds 


6 




48 


Maintaining accountability 


5 


4 


50 


Disseminating Info/Outputs 


4 


4 


18 


Presentations 


f 


1 


19 


Benchmark Decisions 


4 


2 


30 


Committee/Consultant Meetings 


3 


1 


15 


Staff Meetings 


2 


— 


16 


Conferences/ Seminars 


2 


1 


39 


Updating cf Machine Systems 


2 


2 


47 


Coordinating Efforts 


2 


1 


49 


Establishing Field Sites 


2 




TOTALS 


130 


65 



Two procedures were followed in an attempt to reduce imbalance. 
First, unequal proportions of the products, events, and conditions iden- 
tified were selected for analysis. Thus, approximately 25 per cent of 
the events, and 70 per cent of the conditions identified were analyzed. 
While this still did not result in a balance between the three categories 
of outputs, it is probable that the numbers arrived at through the pro- 
portioning process represent a closer approximation to the actual distri- 
bution of such outputs within the domain than would otherwise have been 
obtained. This was judged to be critical from the point of view of 
collecting work requirement data. 

The second procedure followed in attempting to reduce the imbalance 
between products, events, and conditions introduced through data collec- 
tion procedures, was an effort to ensure that the range of outputs analyzed 
was representative of those identified, and that the outputs selected 
for analysis merited analyzing. In this regard, three decision rules 
guided the selection of specific outputs to be analyzed: ( a ) outputs 

judged to be critical to the operation and/or success of a project by the 
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TABLE 8.3 



The Distribution of Outputs Identified and Analyzed 
By Co. dition Clusters 



Set 

DE 

Coding 


Cluster of Outputs: 
Conditions 


Frequency of Outputs 




Identified 


| Analyzed 




21 


Working Environment /Atmosphere 


19 


15 




20 


Cooperative Relationships 


14 


12 




34 


Coordinated Efforts 


10 


5 




22 


Quality Assurance 


7 


5 




23 


Accountability 


6 


5 




25 


Resource Accessibility 


4 


1 




41 


Effective Communication 


4 


1 




44 


Provision for Services 


4 


3 




45 


Informed Public 


2 


1 




46 


Trained Staff 


1 


1 




35 


Acceptance/Adoption of Outputs 


1 


1 




26 


Available Field Sites 


1 


1 




TOTALS 


73 


51 





staff of that project were given selection priority; (b_) outputs that were 
unique relative to the pool of outputs already analyzed at any point in time 
wore given selection priority; and (jc) the number of outputs selected for 
analysis within any given output cluster category was to reflect the total 
number of outputs identified within that category. Table 8.4 presents the 
distribution of outputs identified and analyzed for each of the 20 projects 
studied. On the basis of the data reported in Tables 8.1 through 8.4 it is 
believed that the sample of outputs selected tor analysis is a reasonably 
good sample of the outputs produced by the 20 projects studied. 



An Overview of the Data Reported 
In the Chapter 



The data reported in this chapter describe the standards, tasks, 
knowledges, skills, and sensitivities associated with th? production of the 
298 outputs analyzed. The data were obtained through on-site interviews 
with 134 professional staff associated with the 20 projects studied. The 
data base includes 1148 descriptions of standards applied to the 298 out- 
puts; 3722 descriptions of the tasks engaged in to produce the 298 outputs 
in accordance with the standards held for them; and 2497 descriptions of 
the knowledges, skills, and sensitivities drawn upon in the production of 
the 298 outputs. The distribution of these descriptive statements, by 
project focus, appears in Table 8,5. The average number of such statements 
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TABLE 8.4 



Tin* I) ( fi t r • l»u t j on nl Outputs Analyzed In IU*l;it Ion 1 41 Outputs 
Identified Within Projects 



Project 
Number a 


Outputs 

Identified 


Outputs 

Analyzed 


Proportion of 
Outputs Analyzed 


i 


40 


10 


.25 


2 


20 


5 


.25 


J 


14 


7 


.50 


4 


28 


5 


.18 


5 


17 


17 


1.00 


6 


•67 


17 


.25 


7 


48 


9 


.19 


8 


15 


9 


.60 


9 


30 


11 


.37 


10 


65 


24 


.37 


11 


22 


11 


.50 


12 


18 


4 • 


.22 


13 


67 


9 


.13 


14 


44 


29 


.70 


15 


70 


23 


.33 


16 


57 


10 


.18 


17 


57 


12 


.21 


18 


47 


12 


.26 


19 


205 


59 


.29 


20 


32 


1j 


.47 



3 

Project numbers listed here coincide with the number given projects in 
the case profiles contained within Volume IV of the series reporting the 
Oregon Studies. 



per output analyzed appears in Table 8.6. In combination, these two tables 
provide the reader with some sense of the data base called upon in the 
analyses presented in the chapter. 

As in the case of project output data (see Chapter 7) task data are 
reported in terms of both primary and cluster categories. The other classes 
of work requirement data did not contain a sufficient number of primary 
categories to warrant clustering. The number of primary and cluster 
categories develooed around standard, task (including cl aster categories), 
and enabler statements, and the application of those categories by project 
foci, are presented in Table 8.7. 
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TABLE 8.5 



The Frequency and Distribution By Project Focus of the 
Standard, Task, and Enabler Statements Elicited 
For the 298 Outputs Analyzed 



Category 

Code 


Variable 


Proiect Focus 


Total 

(20) 


Res 

(5) 


Dev 

V) 


Diff 

SI L 


Eval 

(3) 


J 


Standards 


136 


453 


399 


160 


1148 


J 1 


Output 


93 


358 


217 


107 


775 


J2 


Pro cesses /Opera t ions 


43 


95 


182 


53 


373 


NOQR 


Tasi'S 


433 


1290 


.1411 


588 


3722 


S 


Enablers 


318 


912 


824 


443 


2497 


S 1 


Knowledges 


148 


325 


326 


165 


964 


S 2 


Skills 


89 


323 


281 


155 


848 


S 3 


Sensitivities 


81 


'64 


217 


123 


685 



TABLE 8.6 

The Average Number (Approximate) of Standard, Task, and 
Enabler Statements Elicited Per Output Analyzed 
Within Each Project Focus 



Category 

Code 


Variable 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(7) 


Eval 

(3) 


Number of Outputs Analyzed 


44 


111 


108 


35 


298 


j 


Standards 


3 


4 


4 


5 


4 


j i 


Output 


2 


3 


2 


3 


2.5 


J 2 


Pro cesses /Operations 


1 


1 


2 


2 


1.5 


NOQR 


Tasks 


10 


12 


13 


17 


12 


S 


Enablers 


7 


8 


8 


13 


8 


S 1 


Knowledges 


3 


3 


3 


5 


3 


S 2 


Skills 


2 


3 


3 


4 


3 


S 3 


Sensitivities 

% 


2 


2 


2 


4 


2 



TABLE 8.7 



The Number of Primary and Cluster Categories Used to Describe 
Work Requirements Within RDD&E Projects 



Category 


Variable 


Number 

of 


No. of Categories Used 


Code 


Categories 

Derived 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


j 


Standards 


79 


42 


58 


68 


44 


ji 


Output 


33 


23 


27 


29 


23 


J2 


Process 


46 


19 


30 


39 


21 


NOQR 

QR 


Tasks 

Primary 


280 


129 


197 


209 


150 


NO 


Cluster 


20 


19 


19 


20 


19 


S 


Enablers 


136 


75 


115 


108 


95 


SI 


Knowledges 


29 


21 


24 


22 


24 


S2 


Skills 


49 


26 


40 


44 


36 


S3 


Sensitivities 


58 


28 


51 


42 


35 



Standards Applied to Outputs 



For each output analyzed in the Oregon Studies, statements were sought 
from interviewees regarding the standards used to judge its adequacy, and 
the adequacy of the processes involved in its production. As a result, two 
sets of standards having significance in the production of each of the 298 
selected outputs were obtained, one pertaining to the adequacy of outputs 
themselves and one pertaining to the adequacy of the processes and opera- 
tions associated with their production. As used in the Oregon Studies the 
concept of standard was defined generally as "a criterion applied to, or 
level of excellence expected of an output." 

To organize the 1148 standards cited, 79 primary categories of stand- 
ards were derived. Of the 79 categories, 33 applied to output standards 
and 46 applied to process standards. The primary categories falling within 
each of these two category sets, and the frequency with which each cate- 
gory within each set was applied to the classification of statements descrip 
tive of standards held within project foci, are displayed in Appendix 4. 

The reader is encouraged to peruse this appendix carefully for it displays 
the full range of standards data identified within the 20 projects studied. 

In the pages that follow attention is directed only to the most fre- 
quently used categories of standards. While such a strategy obviously 



ignores much that is contained in Appendix 4, it has been adopted on the 
basis of economy and the assumption that the most meaningful search of 
the data for similarities and differences by project focus rests with 
categories that have a high frequency of use. The data reported in the 
following paragraphs are reported separately for output and process 
s tandards . 



Output Standards 

Table 8.8 displays the five most frequently appearing categories of 
output standards, and the frequencies with which they were applied within 
project foci. The fact that 45 per cent of all standards identified were 
classified within five categories, and that 38 categories were required 
to classify the remaining standards statements, gives an indication of 
the range of standards that exist. 

TABLE 8.8 

The Five Categories Used Most Frequently to Classify the Standards 
Applied to Outputs of Educational RDD&E Projects 



Set LM 


Primary Categories of 




Project locus 


Total 


Coding 


Output Standards 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


(20) 


12 


Goal attainment 


18 


35 


• 31 


18 


102 


13 


Acceptance by others (in proj) 


10 


38 


14 


15 


77 


01 


Completeness of content 


10 


29 


j.8 


3 


60 


06 


Acceptance by users 


0 


19 


35 


3 


57 


05 


Utility or value 


1 


28 


19 


7 


55 


Total number of output standards coded 
with the five categories 


39 


149 


117 


46 


351 


Total number of output standards 


93 


358 


217 


107 


775 


Percentage of output standards coded 
with the five categories 


42% 


42% 


54% 


43% 


45% 



The data included in Table 8.8 gives evidence of the existence of both 
similarities and differences in the standards applied to outputs across 
RDD&E. Similarities are evidenced by the fact that the five categories of 
standards cited accommodated about the same proportion of standards state- 
ments identified within all project foci. Also, personnel in all project 
foci appear to be concerned with achievement of goals and the acceptance 
of work by others in the project. Beyond this, however, similarities across 
project foci appear to diminish rapidly. 
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As might be expected, personnel in diffusion projects are concerned 
with the acceptance of their outputs by users. By contrast, personnel 
in research projects, and, to a lesser extent, evaluation projects, appear 
relatively unconcerned about whether their products are accepted by users. 
While personnel in all other foci place some importance on the utility or 
value of their outputs, personnel in research projects appear relatively 
unhampered by such standards. 

Table 8.9 provides a closer look at the standards considered most 
critical within educational RDD&E projects. Again, the five most fre- 
quently used categories of output standards are presented, but this time 
they are listed for each project focus separately. As would be expected, 
the data in this table tend to reflect those in Table 8.8. Several new 
standards appear in Table 8.9, however, which add clarity to the nature 
of the standards applied to outputs by personnel in differing project 
areas . 

One of the output standards listed in Table 8.9 that does not appear 
in Table 8.8 is that of Personal Satisfaction/Feeling. This was held by 
personnel in both research and development projects. A standard held only 
by researchers had to do with an output Functioning as Planned. It is 
interesting that while research personnel seem to pay little attention to 
standards of utility or acceptance by users, they do attend to standards 
that pertain to feelings of satisfaction and having outputs function as 
planned . 

Personnel in evaluation projects appear to rely more upon standards 
that reflect the appropriateness of or accuracy in their efforts than do 
other project personnel. Categories of standards appearing in Table 8.9 
for evaluation projects that did not appear in Table 8.8, relate to the 
precision with which outputs are produced (Lack of Errors/Discrepancies) 
and the importance of appropriate design and content for the production 
of cutputs. The standards most frequently cited by personnel working 
within diffusion projects are the same as those listed in Table 8.8. 

One further analysis of the categories used most frequently to classi- 
fy output standards is presented. In this analysis the categories of 
standards appearing in Tables 8.8 and 8.9 have been combined, and the per- 
centage of the total set of standards occurring within them, by project 
focus, has been calculated. These data are displayed in Table 8.10. 

Two observations about :he data presented in Table 8.10 merit consid- 
eration. First, the standards added to those appearing in Table 8.8 
account for a sizable iucrease in the percentage of standards statements 
classified. This is especially the case in evaluation projects where the 
percentage moves from 45 to 82 per cent. Even with the added standards, 
however, a substantial proportion of the standards statements identified are 
classified by other categories. This again speaks to the range of standards 
data and reinforces the desirability of carefully reviewing Appendix 4. 

The second observation is that the four added standards appear to have mean- 
ing within all project foci. In only one instance did any of the added 
categories of standards account for the classification of less than three 
per cent of the total set of standard statements identified within project 
focus • 



TABLE 8.9 



The Five Most Frequently Used Categories of Output Standards 
Within Project Foci, Their Frequency of Use, and the 
Percentage of Standards Statements Coded Within Them 



Set LM 
Coding 


Primary Categories of 
Output Standards 


Freq 

of 

Use 


% of Stand- 
ards State- 
ments Coded 


Research Focus (5 Projects) 


12 Goal attainment 

13 Acceptance by others (in project) 

01 Completeness of content 

07 Personal satisfaction or feeling 

22 Functions as planned i 


18 19% 

10 11 

10 11 

8 9 

6 6 


Totals (5 categories) 


52 56% 



Development Focus (7 Projects) 



13 


Acceptance by others (in project) 


38 


10 


12 


Goal attainment 


35 


10 


01 


Completeness of content 


29 


8 


05 


Utility or value 


28 


8 


07 


Personal satisfaction or feeling 


28 


8 


Totals (5 categories) 


158 


44% 



Diffusion Focus ( 5 Projects) 



06 


Acceptance by users 


35 


16 


12 


Goal attainment 


31 


14 


05 


Utility or value 


19 


9 


01 


Completeness of content 


18 


8 


13 


Acceptance by others (in project) 


14 


7 


Totals (5 categories) 


117 


54% 



Evaluation Focus (3 Projects) 



12 


Goal attainment 


18 


16 


13 


Acceptance by others (in project) 


15 


14 


11 


Appropriate design/content 


10 


9 


05 


Utility or value 


7 


7 


09 


Lack of errors/discrepancies 


7 


7 


Totals (5 categories) 


57 


53% 




141 

123 



TABLE 8.10 



The Nine Categories Used Most Frequently to Classify 
Standards Applied to the Outputs of Educational RDD&E, 
and the Percentage of Standards Statements 
Coded Within them by Project Focus 



Set LM 


Primary Categories of 


Project Focus 




Coding 


Output Standards 


Res. 

(5) 


Dev. 

(7) 


Diff . 
(5) 


Eval. 

(3) 


Total 

(20) 


12 


Goal Attainment 


19% 


10% 


14% 


16% 


13% 


13 


Acceptance by others (in proj) 


11 


10 


7 


14 


10 


01 


Completeness of content 


11 


8 


8 


6 


8 


06 


Acceptance by users 


0 


5 


16 


6 


7 


05 


Utility or value 


1 


8 


9 


7 


7 


07 


Personal satisfaction/feeling 


9 


8 


4 


4 


6 


09 


Lack of errors/discrepancies 


3 


6 


3 


7 


5 


22 


Functions as planned 


6 


4 


4 


13 


5 


11 


Appropriate design/content 


1 


3 


5 


9 


4 


Percentage of all output standards 
included within these categories 


61% 


62% 


70% 


82% 


65% 



Process Standards 



Process standards are treated in the paragraphs that follow in the 
same manner that output standards have been treated. Table 8.11 contains 
the five most frequently used categories of process standards. Table 8.12 
contains the five most frequently used categories within each project focus, 
and Table 8.13 displays the percentage of standards classified within each 
project focus by the categories listed in Tables 8.12 and 8.13. 

As can be seen from Table 8.11, there does not appear to be as much 
sharing of process standards across project foci as was the case with out- 
put standards. With the exception that the five categories account for 
approximately the same proportion of classifications within each project 
focus (see the bottom line of the table) the process standards that person- 
nel in one focus hold as important are held to be either more or less 
important in other foci. With the possible exception of deadlines being 
met, no single process standard appears to be of equal or similar impor- 
tance across foci. 

The data presented in Table 8.12 add further clarity to this point, 
for seven new categories of standards appear within it. Furthermore, of 
the 12 process standards listed in the table, only that of meeting dead- 
lines occurs prominently in as many as three foci. No other process stand- 
ard appearing in the table is shared by more than two project foci. 
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TABLE 8.11 



The Five Categories Used Most Frequently to Classify 
the Standards Applied to Processes in 
Educational RDD&E Projects 



Set LM 


Primary Categories of 


Project Focus 


Total 


Coding 


Process Standards 


Res . 
(5) 


Dev. 

(7) 


Diff . 
(5) 


Eval. 

(3) 


(20) 


34 


Impact of effort favorable 


3 


5 


16 


10 


34 


17 


External enthusiasm evident 


6 


4 


20 


2 


32 


04 


Deadlines are met 


5 


9 


9 


6 


29 


13 


Work conducted within budget 


2 


7 


18 


1 


28 


36 


Employment criteria met I 


! 1 


6 


17 


0 


24 


Total number of process standards 
coded with the five categories 


17 


31 


80 


19 


147 


Total number of process standards 


43 


95 


182 


53 


373 


Percentage of process standards 
coded with the five categories 


40% 


33% 


44% 


36% 


39% 



In reviewing the data in Table 8.12 at another level of detail, it 
can be seen that personnel in research projects emphasize four process 
standards that are emphasized by personnel in other foci. Surprisingly, 
in light of the output standards discussed previously, research person- 
nel hold process standards relating to external enthusiasm to be of major 
importance. This seems somewhat contradictory in light of their apparent 
lack of concern with output standards relating to acceptance by users. 

Not so surprisingly, however, the concern for external enthusiasm is 
shared by personnel in diffusion projects, and seemingly with the same 
degree of emphasis. Research personnel, like development and evaluation 
personnel, are also concerned with meeting deadlines, with achieving an 
acceptable level of output, and with ensuring that an expected activity 
occurs. Based on the data presented in the table, research personnel 
uniquely emphasize cooperation of personnel as a standard of excellence 
to be applied to project operations. 

Personnel in development projects share with personnel in diffusion 
projects sets of standards related to staying within the budget, and with 
seeing to it that personnel being hired meet appropriate criteria. The 
uniqueness of development personnel lies in their willingness to use as 
a standard for judging process feelings that no deficiencies exist in 
that which they are doing. 



TABLE 8.12 



The Five Mos'. Frequently Used Categories of Process Standards 
Within Project Foci, Their Frequency of Use, and the 
Percentage of Standards Statements Coded Within Them 



Set LM 
Coding 


Primary Categories of 
Process Standards 


Freq 

of 

Use 


% of Stand- 
ards State- 
ments Coded 


Research Focus (5 Projects) 





17 


External enthusiasm evident 


6 


14 


04 


Deadlines are met 


5 


12 


01 


Personnel cooperate 


3 


7 


05 


Acceptable level of output 


3 


7 


07 


An expected activity occurs 


3 


7 


Totals (5 Categories) 


20 


47% 



Development Focus (7 Projects) 



04 


Deadlines are met 


9 


10 


13 


Work conducted vithin budget 


7 


7 


05 


Acceptable level of output 


6 


6 


36 


Employment criteria met 


6 


6 


14 


No felt deficiencies 


5 


5 


Totals (5 Categories) 


33 


34% 



Diffusion Focus (5 Projects) 



17 


External enthusiasm evident 


20 


11 


13 


Work conducted within budget 


18 


10 


36 


Employment criteria met 


17 


9 


15 


Tasks perceived and acted upon 


16 


9 


34 


Impact of effort favorable 


16 


9 


Totals (5 Categories) 


87 


48% 



Evaluation Focus (3 Projects) 





34 


Impact of effort favorable 


10 


19 




04 


Deadlines are met 


6 


11 




32 


Resources (info) avail on request 


5 


9 




07 


An expected activity occurs 


4 


8 




35 


Outputs distributed/requested 


4 


8 


Totals (5 Categories) 


29 


55% 




126 
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Personnel in diffusion projects once again reflect standards held 
in common by all other RDD&E project personnel studied, that is, the 
standards listed for diffusion projects in Table 8.12 are with one excep- 
tion the standards appearing in Table 8.11. Only in their reliance upon 
the ability to perceive and act upon tasks involved in their work do diffu- 
sion personnel differ from personnel in other project foci in the process 
standards held. 

Personnel in evaluation projects appear most unique in terms of the 
process standards they act upon. They emphasize two standards that are 
not emphasized by others, and, for the standards that they do share with 
others, the degree of emphasis differs. The two sets of standards emphasized 
by evaluation personnel and not others are, having available to users at 
the time it is needed materials and/or information they require, and having 
outputs requested and distributed. By a considerable margin, however, 
standards relating to the favorable impact of their efforts are of greatest 
concern . 

The data presented in Table 8.13 reinforce the differences observed 
in Table 8.12 with respect to the process standards held across project foci. 



TABLE 8.13 

The Twelve Categories Used Most Frequently to Classify the 
Process Standards Applied in Educational RDD&E, and 
the Percentage of Standards Statements 
Coded Within Them By Project Focus 



Set LM 
Coding 


Primary Categories of 
Process Standards 


Proiect Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


Df.ff 

(5) 


Eval 

(3) 


34 


Impact of effort favorable 


7% 


5% 


9% 


19% 


9% 


17 


External enthusiasm evident 


14 


4 


11 


4 


9 


04 


Deadlines are met 


12 


10 


5 


11 


8 


13 


Work conducted within budget 


5 


7 


10 


2 


8 


36 


Employment criteria met 


2 


6 


9 


0 


6 


15 


Tasks perceived and acted upon 


0 


2 


9 


0 


5 


05 


Acceptable level of output 


7 


6 


2 


6 


4 


07 


An expected activity occurs 


7 


1 


2 


8 


3 


14 


No felt deficiencies 


2 


5 


3 


0 


3 


01 


Personnel cooperate 


7 


1 


2 


2 


2 


32 


Resources available on request 


0 


0 


1 


9 


2 


35 


Outputs distributed/requested 


0 


0 


1 


8 


2 


Percentage of all process standards 
included within these categories 


63% 


47% 


64% 


69% 


61% 



In summarising the standards data, the finding that is most striking 
is the contrast between the similarity in standards emphasized by RDD&E 
personnel in relation to outputs and the difference in standards empha- 
sized in relation to processes and operations. RDD&E personnel working 
within differing project foci seem to apply approximately the same stan- 
dards to the outputs of their work effort, but considerably different 
standards to the processes involved in producing those outputs. It is 
unclear why this should be so, unless for some reason the outputs produced 
within varying project foci involve sufficiently different operations for 
their production that different process standards have to be brought to 
bear. In light of the data reported in the next section of the chapter, 
however, this does not appear to be a particularly promising hypothesis. 



Tasks Involved in Output Production 



r 

As used in the Oregon Studies, a task is defined as "the unit of work 
performed in producing a specified output to a specified standard.” In 
terms of the conceptual framework that guided data collection within tho 
Studies, tasks ware seen as one of three hierarchically ordered descriptors 
of work effort. The largest or most inclusive descriptor was labeled 
activity , the next largest, task , and the next, action . Operationally 
this three-level classification scheme reflects the simple recognition that 
work effort may be treated in larger or smaller units depending upon the 
size of the output chosen for analysis, and that it would be desirable to 
eliminate as much of the arbitrariness in that variability as possible. 

For a number of reasons such distinctions were never made in the Oregon 
Studies and "tasks" became the only unit of analysis used in describing 
wurk effort within a project. 



Two factors prevented the description of work effort at the level of 
activities, tasks, and actions: (a) the inability to arrive at satisfac- 

tory decision rules for determining size or level of output as a starting 
point for the analysis of work effort; and (b) the lack of a resource base 
that would permit the analysis of work effort at an extremely detailed 
level, i.e., the level of action. Given these constraints the decision 
was made to permit activities, tasks, and actions to be differentiated 
empirically rather than conceptually. Operationally, the plan was to involve 
the collection of descriptions of work effort, under the label of tasks, 
without regard to the inclusiveness of the work effort being described, and 
then analyze the total pool of task statements with a view to sorting them 
on the basis of their level of generality. Labels such as activities, tasks, 
and actions — or come other set — could then be applied to the differentiated 
levels of work activity. The data collection phase of the plan has been 
completed; the follow on analysis has not. 




The concept of task, as used in the Oregon Studies, involves two 
additional elements that are critical to its understanding: The linkage 

of tasks to outputs, and the influence that standards held for outputs have 
upon tasks to be performed. Tasks are always identified in relation to the 
production of a particular output, and the nature of the tasks to be per- 
formed are always a function of the standards held for that output. The 
tasks required to develop a test to measure understanding of human develop- 
ment, for axample, are quite different from the tasks involved in developing 
a conferwice designed to lead to increased understanding of human develop- 
ment. In addition, if the standards held for a test are rigorous, for 
example, evidence of item discrimination, evidence of stability of measure- 
ment over time, etc., as opposed to being less rigorous, the tasks to be 
engaged in vary considerably. If the standards for a conference are such 
that participants must show evidence of learning gains during the conference, 
show evidence of activities that follow from the conference, etc., the tasks 
involved in the development and operation of the conference are much differ- 
ent than if less stringent standards were held.-* 

The data on tasks represents the largest single set of data within the 
Oregon Studies. Three thousand, seven hundred twenty-two separate state- 
ments describing tasks performed in relation to the 296 outputs analyzed 
were developed to classify the task statements in terms of their similari- 
ties and differences at a ’’close to the source” level (primary categories), 
and, 20 further categories were developed to classify the 380 primary 
categories (cluster categories). Both the primary and cluster category 
sets were developed for ta ;ks in much the same way as were categories for 
outputs (see Chapter 4, pp 53 to 56). ^ A full listing of the primary task 
categories ordered by the task clusters in which they fall appear as 
Appendix 6. The 20 task cluster categories, the number of primary task 
categories contained within each cluster category, and the number of task 
statements that have been coded as falling within each cluster category, 
are presented in Table 8.14. 



^ As indicated previously, while such interaction exists between outputs, 
standards, and tasks, the data presented in the present chapter on tasks 
are presented as if they were independent of these interactions. The reader 
needs to interpret the data accordingly. 

^ A difference that exists in the category sets for tasks and outputs, how- 
ever, is the nature of the linkage between primary and cluster categories 
within each set. As presently constructed, primary task categories are 
restricted to a single task cluster. This is not the case with outputs, for 
there primary output categories can link to any cluster category. The fixed 
relationship between primary and cluster categories of tasks . function 
of beginning the category development process for tasks with preconceived 
category sets, and the accompanying decision rule that required all primary 
categories to be linked to a specific cluster category. 



TABLE 8.14 



The Number of Task Statements and Primary Task Categories 
Contained Within Each Cluster Task Category 



Set 

NO 

Coding 


Task Cluster Categories 


Number of 
Primary Task 
Categories 


Number of 
Task 

Statements 


04 


Producing the output 


17 


474 


05 


Collecting/processing data 


19 


342 


06 


Assessing the output quality 


17 


337 


22 


Effecting accountability 


32 


314 


03 


Designing the output 


20 


300 


24 


Effecting quality control 


24 


298 


01 


Clarifying problem addressed 


16 


297 


29 


Facilitating relationships 


18 


225 


31 


Diffusing info, within project 


15 


205 


21 


Procuring professional staff 


19 


201 


23 


Procuring systems /services 


9 


199 


02 


Formulating objectives 


7 


141 


25 


Maintaining job satisfaction 


15 


124 


33 


Effecting decision mechanisms 


10 


67 


30 


Effecting info, flow patterns 


8 


61 


26 


Facilitating growth of staff 


6 


47 


32 


Diffusing info, beyond project 


11 


40 


07 


Diffusing the output 


10 


36 


27 


Enhancing physical environment 


4 


8 


28 


Maintaining equity among staff 


3 


6 


Totals 


280 


3722 



As can be seen from the table, task cluster categories tend to be 
extremely general in nature, and tend to subsume a relatively large number 
of primary task categories and task statements. While useful for purposes 
of organizing large quantities of data, their generality and inclusive- 
ness make them poor vehicles for displaying the nature of the task data 
collected in the Studies. Because of the relatively large frequencies 
involved; the primary task categories make better display vehicles. In 
keeping with the strategy of adopting high frequency categories for pur- 
poses of cross-project analysis and discussion, as was done with standards, 
the ten most frequently used categories of tasks across all projects studied 
are presented in Table 8.15. 
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TABLE 8.15 



The Ten Categories Used Most Frequently to Classify the Tasks 
Engaged in Within Educational RDD&E Projects 



Set 
NO QK 
Coding 


Primary Categories of 
Tasks 


Project Focus 


Total 

(20) 


Res . 
(5) 


Dev. 

(7) 


DiJf . 
(5) 


Eval . 
(3) 


0A 02 


Construct/assemble components 


17 


97 


75 


36 


225 


06 14 


Note/make modifications 


7 


90 


A7 


1A 


158 


01 12 


Review existing materials 


12 


A2 


AA 


12 


110 


23 05 


Negotiate contrac t/coopsrat ion 


7 


22 


63 


3 


95 


05 02 


Administer data instruments 


12 


22 


27 


8 


79 


0A 05 


Produce the output 


7 


31 


29 


10 


77 


03 10 


Specify/identify charac of mat 


A 


A3 


21 


9 


'll 


02 07 


Confer w/colleagues re obj 


11 


35 


16 


15 


77 


25 01 


Specify/explain agcy/project psn 


0 


2 


63 


A 


69 


2 A 11 


Proof /critique products/perform 


2 


16 


3A 


9 


61 


Total number of task statements 


7 q Ann 


A 9Q 


i on 




coded within the ten categories 


/ y 4UU 




izu 


1UAO 


Total number of task statements 


A33 1290 1411 


588 


3722 


Percentage of task statements 


18% 


31% 


30% 


20% 


28% 


coded within ten categories 













It will be seen from the data presented in the table that the ten 
categories used most frequently to classify the tasks engaged in within 
educational RDD&E do not begin to represent the range of tasks identified 
by the Oregon Studies, In no project focus do the ten categories account 
for as much as a third of the tasks identified. It will also be seen that 
the task categories listed receive greater use in classifying work efforts 
within development and diffusion projects than within research and evalua- 
tion projects, a fact accounted for in large part by the absolute number of 
task statements identified within development and diffusion projects. 

Because of the extensiveness of the task data collected (another 270 primary 
task categories used in coding 72% of the task statements are not mentioned 
in Table 8.15;, and because of the wide variation of tasks performed within 
RDD&E projects, the reader is encouraged to peruse Appendix 6 carefully. 

Only a glimpse of the data within it can be provided in the next few pages. 



The most frequently cited task category in all project foci was 
that of constructing or assembling the components of an output. As used 
within the Oregon Studies, this category pertains to the fabrication of an 
output, or its pieces and parts. It refers to the somewhat mechanical 
ta3k of actually putting an output together after all else in its develop- 
ment has been dona. As such, it f s an activity common to all project foci. 

The next most, frequently appearing task, and one that is closely 
related to the first, is that of modifying an output. As would be expec- 
ted, the category was most frequently used in classifying work effort 
with 1 * . development and diffusion projects, but it also appears in research 
and /aluation projects. When these two categories are combined with the 
task '’Producing the Output,” i.e., making multiple copies of an output 
available, it is clear that a great deal of work effort is directed in 
RDD&E projects, especially development and diffusion projects, to an out 
and out production function. This becomes even more apparent if the tasks 
Review Materials, Specify the Character of Materials to be Produced, and 
Proof or Critique a Product and/or its Performance are added to the list. 

It is true of course that the data reported in Table 8.15 are influenced 
heavily by the preponderance of the tasks reported being identified within 
development and diffusion projects, but the data hold reasonably well 
across all project foci. 

With one exception the data presented in Table 8.15, at a gross level, 
pattern reasonably well. Except for the category Explaining the Position 
of a Project or Agency to Others, all ten categories tend to be used more 
frequently in classifying work activities identified in development and 
diffusion projects than in research and evaluation projects. That is the 
case both for the absolute number of times used and the proportion of work 
activities classified. Within the limits of the data then, it is possible to 
say that personnel in development and diffusion projects tend to engage in simi- 
lar kinds of work activities, and that personnel in research and evaluation 
projects tend to engage in similar kinds of work activities. All four foci 
engage in common tasks but with differing degrees of emphasis. 

Within this gross pattern, however, there is a great deal of varia- 
bility, In six of the ten categories, for example, the difference in 
frequency of use between development and diffusion projects is half or 
more. The same statistic holds for research and evaluation. Thus, while 
development and diffusion seem to share reasonably common work require- 
ments at one level, and research and evaluation seem to do the same, at 
another level all foci seem to engage in the execution of tasks that assume 
somewhat idiosyncratic patterns. The reader is urged to pursue such analyses 
by use of the date presented fn Appendix 6. 

Table 8.16 provides a closer look at the work activities most fre- 
quently engaged in within educational RDD&E. In this table the five 
categories used most frequently to classify work activities identified 
within project foci are displayed. As would be expected from the data in 
Table 8.15, no new categories of work activity appeared in Table 8.16 for 
development and evaluation projects. The five most frequently used cate- 
gories of work effort within both of these areas were subsumed within the 
ten most frequently used categories within the domain as a whole; at least 



TABLE 8. lb 



The Five Host Frequently Used Categories of Tasks Engaged in 
Within Project Foci, Their Frequency of Use, and the 
Percentage of Task Statements Coded V.’ithin Thera 



Set 
NO QR 
Coding 


Primary Categories 
of Tasks 


Freq. 

of 

Use 


% of Stand- 
ards State- 
ments Coded 


Research Focus (5 Projects) 


04 02 Construe t/asserable components 

05 02 Administer data instruments 

01 12 Review existing materials 

02 07 Confer w/colleagues re objectives 

29 14 Interact w/setting to collaborate 


17 4% 

12 3 

12 3 

11 2 

9 2 


Totals (5 categories) 


61 14% 



Development Focus (7 Projects) 



04 02 


Construct/assemble components 


97 


8% 


06 14 


Note/make modifications 


90 


7 


03 10 


Specify/identify charac of materia] 


43 


3 


01 12 


Review existing materials 


42 


3 


02 07 


Confer w/colleagues re objectives 


135 


3 


Totals (5 categories) 


407 


24% 



Diffusion Focus (5 Projects) 



04 02 


Construct/assemble components 


75 


5% 


23 05 


Negotiate contract/cooperation 


63 


5 


25 01 


Specify/explain agcy/project psn 


63 


5 


06 14 


Note/make modifications 


47 


3 


01 12 


Review existing materials 


44 


3 


Totals (5 categories) 


292 


21% 



Evaluation Focus (3 Projects) 



04 02 


Construct/assemble components 


36 


6% 


02 07 


Confer w/colleagues re objectives 


15 


3 


06 14 


Note/make modifications 


14 


2 


04 01 


Construct/assemble msmt tools 


14 


2 


22 04 


Establish/maintain time lines 


13 


2 




Totals (5 categories) 


92 


15% 
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as the domain was sampled by the 20 projects studied. New categories 
descriptive of work effort within research and evaluation projects do 
appear in Table 8.16, however. The category Interacting with Persons in 
a Setting to Obtain their Collaboration, and the categories Constructing 
Measurement Tools and Establishing and Maintaining Time Lines, reflect 
work efforts that appear more frequently in research and evaluation projects, 
respectively, than within the domain of RDD&E as a whole. Interestingly, 
the column in Table 8.16 showing percentages indicates that no task cate- 
gory accounts for more than eight per cent of the task statements identified 
within any project focus, and that by the time the fifth most frequently 
used category is displayed the per cent of the task statements incor- 
s porated dwindles to three per cent or less. When the 13 different cate- 
gories that appear in Tables 8.15 and 8.16 are combined, as they are in 
Table 8.17, the percentage of task statements they account for diminishes 
even more dramatically. Considering the large number of task categories that 
remain unreported (267), and the percentage of task statements they incor- 
porate, the range of work activities encountered within the domain of 



TABLE 8.17 

The Thirteen Categories Used Most Frequently to Classify the Tasks 
Engaged in Within Educational RDD&E, and the Percentage 
of Task Statements Coded Within Them By Project Focus 



Set 
NO QR 
Coding 


Primary Categories 
of Tasks 


Project Focus 


Total 

(20) 


Res . 
(5) 


Dev. 

(7) 


Diff . 
(5) 


Eval. 

(3) 


04 


02 


Cons true t/ assemble components 


4 


8 


5 


6 


6% 


06 


14 


Note/make modifications 


2 


7 


3 


2 


4 


01 


12 


Review existing materials 


3 


3 


3 


2 


3 


23 


05 


Negotiate contrac t/cooperation 


2 


2 


4 


1 


3 


05 


02 


Administer data instruments 


3 


2 


3 


1 


2 


04 


05 


Produce the output 


2 


2 


2 


2 


2 


03 


10 


Specify/identify charac of matter 


1 


3 


1 


2 


2 


02 


07 


Confer w/colleagues re objectives 


3 


3 


1 


3 


2 


25 


01 


Spec if y/explain agcy/project psn 


0 


- 


4 


1 


2 


24 


11 


Proof /critique produc ts/perform 


- 


1 


2 


2 


2 


29 


14 


Interact v/setting to collaborate 


2 


1 


2 


2 


1 


22 


04 


Establish/maintain time lines 


4 


1 


- 


2 


1 


04 


01 


Cons true t/assemble msmt tools 


1 


1 


1 


2 


1 


Percentage of all task statements in- 
cluded within these categories 


27% 


34% 


31% 


28% 


31% 





134 




educational RDD&E can begin to be appreciated. Again, the reader is urged 
to study Appendix 6 carefully in this regard.^ 



Knowledges, Skills, and Sensitivities 
Called Upon in Output Production 

Within the context of the Oregon Studies, knowledges, skills, and 
sensitivities were seen as "enablers" of task performance. As such, they 
were seen as prerequisites to output production and, for purposes of economy 
in data collection, were linked directly to outputs.® As used in the 
Oregon Studies knowledge, skill, and sensitivity were defined as follows: 

Knowledge - A classification given an enabler that identi- 
fies it as a fact, principle, or generalization 
that can stand the test of empirical verifica- 
tion; also, any circumstance that can be shown 
to exist. 



Skill 



A classification given an enabler that identifies 
it as an ability, proficiency, or expertness in 
the exercise of an art, craft, or science. 



With the range and variability that exist in work activities (tasks) 
within the domain of educational RDD&E, some means must be found to 
cluster them meaningfully. Two such efforts are reported in the present 
volume: 00 the cluster categories that classify the 230 primary categories 

of work activity (see Table 8.14, p. 130); and (b) the 3.inkage of work 
activities to outputs (see Chapters 10 and 11). A number of additional 
analyses could be undertaken in this regard now that the task data are avail- 
able in final form. These should include formal factorial studies, further 
conceptual analyses, and the completion of the output work requirement 
analyses introduced in Chapters 10 and 11. 



8 

This was recognized as a grosser linkage than desired, but because of the 
extremely large number of task statements identified, time and resource 
limitations, and the tendency to find the same enablers serving multiple 
tasks, the decision was made to link enabler data directly to outputs rather 
than to the tasks engaged in to produce outputs. One consequence of this 
decision was the problem it posed in distinguishing between task and skill 
statements. Since task statements were always elicited from interviewees 
first, however, and the same interviewees provided the skill data, it was 
possible to eliminate overlap between such statements with respect to any 
particular output. When looking at task and skill statements across out- 
puts, however, some overlap appears. The pros and cons of linking enablers 
to tasks as opposed to outputs should be investigated further. 
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Sensitivity - A classification given an enabler that identi- 
fies it as an increment of awareness about an 
environment or factors operating in or upon an 
environment; also, attitudes and personality 
characteristics . 



Examples of knowledge include standard school subjects, the subject matter 
of RDD&E , resource availability, and staff responsibilities; examples of 
skill include writing, facilitating people interactions, using equipment 
or systems, and exercising judgment; and examples of sensitivity include 
an awareness of staff capabilities and limitations, awareness of needs of 
self and others, awareness of potential conflict of interests, and feel- 
ings of personal security. 

A total of 2,497 enabler statements, as opposed to 3,722 task state- 
ments, were collected. 9 Of this total 964 were knowledge statements, 848 
were skill statements, and 685 were sensitivity statements. These state- 
ments, in turn, were grouped into 29 primary categories of knowledge, 49 
primary categories of skill, and 58 primary categories of sensitivity. 
These three sets of categories appear, respectively, in Appendices 7, -8, 
and 9. No cluster categories were developed for any of the three. The 
percentage distribution of the three classes of enabler statements, by 
project focus, is shown in Table 8.18. 



TABLE 8.18 

The Percentage Distribution of Knowledge, Skill, and Sensitivity 
Statements Within Project Foci 



Set S 
Coding 


Structure 
of Enabler 


Project Focus 


Total 

(20) 


Res. 

. (5)_ 


Dev. 

(7) 


Diff . 
(5) 


Eval . 

(31 


1 


Knovrledge 


47% 


36% 


40% 


37% 


39% 


2 


Skill 


28 


35 


34 


35 


34 


3 


Sensitivity 


25 


29 


26 


28 


27 


TOTALS 


100% 


100% 


100% 


100% 


100% 



It is somewhat surprising that the number of enabler statements was less 
than the number of task statements. It is unclear whether this was a func- 
tion of the way in which enabler statements were identified, that is, linked 
directly to outputs rather than the tasks engaged in to produce outputs, or 
whether it was a function of a limited set of knowledges, skills, and sensi- 
tivities being drawn upon in multiple combinations in the performance of a 
wide range of tasks. From a training standpoint this becomes a critical 
issue, and in subsequent studies should be resolved. 
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Enabling Knowledges 



To continue the pattern of analysis established in the previous 
pages, the five categories most frequently used to classify the knowledges 
drawn upon in the 20 projects studied are displayed in Table 8.19, Table 
8.20 displays the five categories of knowledge most frequently drawn upon 
within each project focus. 



TABLE 8.19 

The Five Categories Used Most Frequently to Classify the Knowledges 
Drawn Upon in Educational RDD&E Projects 



Set UV 
Coding 


Primary Categories of 
Enabling Knowledges 


Project Focus 


Total 

(20 


Res 

(5) 


Dev 

(7) 


Diff j 

(5) | 


| Eval 
1 (3) 


06 


Project variables: external 


29 


38 


81 


9 


157 


03 


Subjects related to RDD&E 


38 


34 


22 


50 


14<n 


04 


Techni cal /prof ess ional topics 


14 


56 


44 


22 


136 


08 


Project operation: specific 


12 


40 


36 


8 


96 


07 


Project operation: general 


7 


22 


19 


5 


53 


Total 


number of knowledge statements 












coded within the categories 


98 


190 


202 


94 


586 


Total 


number of knowledge statements 


148 


325 


326 


165 


964 


Percentage of knowledge statements 












coded with the five categories 


68% 


58% 


62% 


57% 


61% 



As can be seen in Table 8.19 knowledge of circumstances that surround 
a project are drawn upon as frequently in project operation as knowledge 
associated with the technical aspects of the field. In combination, the 
three categories of knowledge appearing in the table that focus upon exter- 
nal and internal conditions affecting project operation are used as fre- 
quently as the two categories of knowledge that focus upon professional 
and discipline related topics. On the basis of the data reported in 
Appendix 7, this is a pattern that characterizes the knowledge data as a 
whole. As such, it has major implications for training, for if taken at 
face value it means that knowledge specific to projects is as critical to 
project operation as is knowledge that is generalizable across projects. 
Because of the generality of the knowledge categories, the reader may wish 
to trace the specific knowledge items incorporated within them by referring 
to the appendices within the case profiles (see Volume IV of the series of 
volumes reporting the Oregon Studies). 



TABLE 8.20 



The Five Most Frequently Used Categories of Knowledge Within 
Project Foci, Their Frequency of Use, and the Percentage 
of Knowledge Statements Coded Within Them 



Set UV 


Primary Categories of 


Freq 


% of Know- 


Coding 


Enabling Knowledges 


of 

— Use 


ledge State- 
ment* 



Research Focus (5 Projects) 



03 


Subjects related to RDD&E 


38 


26 


06 


Project variables: external 


29 


20 


04 


Technical/professional topics 


14 


9 


08 


Project operation: specific 


12 


8 


05 


Project focus topics: external 


11 


7 

• 


Totals (5 Categories) 


104 


70% 



Development Focus (7 Projects) 



04 


Technical/professional topics 


56 


17 




08 


Project operation: specific 


40 


12 




06 


Project variables: external 


38 


12 




03 


Subjects related to RDD&E 


34 


10 




07 


Project operation: general 


22 


7 




Totals (5 Categories) 


190 


58% 





Diffusion Focus (5 Projects) 



06 


Project variables: external 


81 


25 


04 


i Technical/professional topics 


44 


13 


08 


Project operation: specific 


36 


11 


03 


Subjects related to RDD&E 


22 


7 


07 


Project operation: general 


19 


6 


Totals (5 Categories) 


202 


62% 



Evaluation Focus (3 Projects) 



03 


Subjects related to RDD&E 


50 


30 


04 


Technical/professional topics 


22 


13 


02 


Subjects learned in courses (gen) 


13 


8 


24 


Process/implementatlon (project) 


'll 


7 


05 


Project focus topics: external 


9 


6 


Totals (5 Categories) 


105 


64% 



While the five knowledge categories listed in Table 8.19 are distri- 
buted across all project foci, different emphases within foci are apparent. 
For both research and evaluation personnel, knowledge of subjects related 
to RDD&E appear to be drawn upon most frequently. Personnel in diffusion 
projects seem to pay particularly close attention to knowledge of factors 
external to project operations, while personnel in development projects 
seem to draw particularly heavily upon technical and professional topics. 

On the whole, however, the data from Table 8.19 suggest that the same 
general classes of enabling knowledges are required of personnel in all 
project foci. This commonality is reported by categories, however, and it 
is necessary to review the data in the profile appendices contained in 
Volume IV to identify possible differences. 

Table 8.20 represents an extension of Table 8.19 in that it displays 
the five categories of knowledge drawn upon most frequently within each 
project focus. Two observations about the data displayed in the table are 
noteworthy: (a) the five categories of knowledge most frequently drawn 

upon in the field as a whole are the five categories most frequently drawn 
upon in development and diffusion projects, although in somewhat different 
order; and (b) personnel in evaluation projects tend to rely more heavily 
upon substantive knowledge, and less heavily upon knowledge of project opera 
tions, than personnel in any other project focus. To some extent research 
personnel shared in this emphasis upon the substantive, as evidenced by the 
appearance in Table 8.20 of the category Project Related Topics, but not so 
heavily as evaluation personnel. The similarities and differences in the 
extent to which project foci rely upon various categories of knowledge are 
highlighted by the data displayed in Table 8.21. 

TABLE 8.21 

The Eight Categories Used Most Frequently to Classify 
the Knowledges Drawn Upon in Educational RDD&E, 
and the Percentage of. Knowledge Statements 
Coded Within Them By Project Focus 



Set UV 


Primary Categories of 


] 


Project Focus 


Total 


Coding 


Enabling Knowledges 


Res . 
, (5) 


Dev. 

( 7 ) 


Diff . 

( 5 ) 


Eval. 
(3) _ 


(20) 


06 


Project variables: external 


20% 


12% 


25% 


5% 


16% 


03 


Subjects related to RDD&E 


26 


10 


7 


30 


15 


04 


Technic al/ prof ess ional topics 


9 


17 


13 


.13 


14 


08 


Project operation: specific 


8 


12 


11 


5 


10 


07 


Project operation: general 


5 


7 


6 


3 


5 


05 


Project focus topics: external 


7 


5 


5 


6 


5 


02 


Subjects learned in courses 


2 


5 


3 


8 


4 


24 


Process/implementation (proj) 


2 


2 


5 


7 


4 


Percentage of all knowledge state- 
ments included within these 


79 % 


70% 


75% 


77% 


73% 


categories 













Enabling Skills 



Table 8.22 contains a listing -of the five most frequently relied upon 
categories of skills in the 20 projects studied. By and large the cate- 
gory labels presented in the table are reasonably self-explanatory. The 
one exception is that of Finding Fits/Integrating. As used in the Oregon 
Studies, this category refers to the integration of ideas, perceiving the 
relevance of something when its relevance is not apparent, fitting people 
to tasks and people to people, etc. 



TABLE 8.22 



The Five Categories Used Most Frequently 
To Classify the Skills Relied Upon 
In Educational RDD&E Projects 



Set UV 


Primary Categories of 


Project Focus 


Total 


Cod ing 


Enabling Skills 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


(20) 


14 


Writing 


12 


35 


32 


11 


90 


02 


Facilitating people interaction 


7 


26 


28 


13 


74 


18 


Finding fits/integrating 


8 


30 


18 


8 


64 


19 


Planning/conceptualizing 


6 


19 


11 


9 


45 


11 


Disciplining self 


5 


17 


7 


9 


38 


Total number of skill statements 
coded within the categories 


38 


127 


96 


50 


311 


Total number of skill statements 


89 


323 


281 


155 


848 


Percentage of skill statements 
coded with the five categories 


43% 


39% 34% 


32% 


37% 



Three features of the data presented in Table 8.22 are noteworthy: 

(a) the nature of the skills listed; (]j) the distribution of their use 
across project foci; and ( c ) the percentage of the total number of skills 
identified that were classified within the five categories. From the point 
of view of the content of most RDD&E training programs, the skills listed 
are a surprise. Writing, Facilitating People Interaction, finding fits 
between people, ideas and things, Planning-Conceptualizing, and disciplin- 
ing oneself to work, are not skills that are commonly trained for. From the 
point of view of persons who have been in the field for any period of time, 
however, and especially those who have directed or managed projects of 
their own, the skills listed are a confirmation of what had long been 
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suspected. The "bread and butter" skills of day-to-day project operation 
and survival in a context of competitive project funding are more basic than 
has heretofore been recognized. 

Interestingly, the five skills seem to be relied upon equally heavily 
within all project foci. In only one instance, that of Disciplining Self, 
did the dominant pattern of use vary by project focus. Compared to other 
work requirement data the most frequently relied upon skills in educational 
RDD&E appear to be reasonably generic to the domain as a whole. 

Compared to the knowledge data just reviewed, the five most frequently 
used categories of skills contain a relatively small percentage of skill 
statements identified in the 20 projects studied. Whereas the five most 
frequently used categories of knowledge accounted for 61 per cent of all 
knowledge statements identified, the five skill categories listed in Table 
8.22 account for only 37 per cent of the skill statements. This suggests 
that the reader should peruse Appendix 8 carefully to determine the full 
range and distribution of skills identified in the 20 projects. Another 45 
categories of skills exist; they contain 63 per cent of the skill state- 
ments collected; and they are distributed much more unevenly across project 
foci than is the case for the five categories presented in Table 8.22, 

Table 8.23 represents an extension of Table 8.22 in that it presents 
the five most frequently relied upon categories of skills within each pro- 
ject focus. Three new categories of skills make their appearance in the 
table: Analytical Data Handling, Persuading/Justifying, and Analytical 

Problem Solving. The category Analytical Data Handling replaces Disciplin- 
ing of Self as a much used skill in research projects; the categories 
Persuading/Justifying and Analytical Data Handling replace Planning/ 
Conceptualizing and Disciplining Self as much used skills in diffusion 
projects; and the category Analytical Problem Solving replaces Planning/ 
Conceptualizing in evaluation projects. By and large, these data are con- 
sistent with task data reported earlier. For example, the proportionately 
high frequency of data collection and reduction tasks in research projects 
and the frequent need to explain or justify diffusion projects to others 
are consistent with the tasks common to those foci. In all project foci, 
however, the skills of Writing and Facilitating People Interactions are 
two of the three most heavily relied upon skills in carrying out project 
operations . 

The data presented in Table 8.24 reconfirm the prominance of these two 
skills in the operation of projects throughout the domain, and begin to 
point to the variability within the skill data as low frequency categories 
are considered. Again, the reader is urged to study Appendix 8 carefully 
for a sense of the range and variability of skills employed in the conduct 
of educational RDD&E. 









141 



¥ 



¥ 



TABLE 8.23 



The Five Most Frequently Used Categories of Enabling Skills 
Within Project Foci, Their Frequency of Use, and the 
Percentage of Skill Statements Coded Within Them 



Set UV Primary Categories of 

Coding Enabling Skills 


Freq % of Skill 

of Statements 

Use Coded 


Research Focus (5 Projects) 


14 Writing 

18 Finding fits/integrating 

02 Facilitating people interactions 

10 Analytical data handling 

19 Planning/conceptualizing 


12 13 

8 9 

7 8 

7 8 

6 7 


Totals (5 Categories) 


40 45% 


Development Focus (7 Projects) 


14 Writing 

18 Finding fits/integrating 

02 Facilitating people interactions 

19 Planning /conceptualizing 

11 Disciplining self 


35 11 

30 9 

26 8 

19 6 

17 5 


Totals (5 Categories) 


127 39% 


Diffusion Focus (5 Projects) 


14 Writing 

02 Facilitating people interactions 

23 Persuading/justifying 

18 Finding fits /Integra ting 

10 Analytical data handling 


32 11 

28 10 

19 7 

18 6 

13 5 


Totals (5 Categories) 


110 39% 


Evaluation Focus (3 Projects) 


02 Facilitating people interactions 

14 Writing 

09 Analytical problem solving 

11 Disciplining self 

18 Finding fits/integrating 


13 8 

11 7 

10 7 

9 6 

8 5 


Totals (5 Categories) 


51 33% 
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TABLE 8.24 



The Eight Categories Used Most Frequently to Classify the Skills 
Relied Upon in Educational RDD&E, and the Percentage of 
Skill Statements Coded Within Them by Project Focus 



Set UV 


Primary Categories of 


Project Focus 


Total 


Coding 


Enabling Skills 


Res . 
(5) 


Dev . 
(7) 


Diff. 

(5) 


Eval . 
(3) 


(20) 


14 


Writing 


13% 


11% 


11% 


7% 


11% 


02 


Facilitating people interaction 


8 


8 


10 


8 


9 


18 


Finding fits/ integrating 


9 


9 


6 


5 


8 


19 


Planning conceptualizing 


7 


6 


4 


6 


5 


11 


Disciplining self 


6 


5 


2 


6 


4 


10 


Analytical data handling 


8 


2 


5 


5 


4 


23 


Persuading/ jus tifying 


4 


2 


7 


2 


4 


9 


Analytical problem solving 


1 


3 


1 


7 


3 


Percentage of all skill statements 
included within these categories 


56% 


46% 


46% 


46% 


48% 



Enabling Sensitivities 

It will be recalled that a sensitivity was defined in the Oregon 
Studies as "an increment of awareness about an environment or factors 
operating in or on an environment, as well as attitudes and personality 
characteristics. " The rationale underlying the collection of data about 
sensitivities was the simple notion that to be successful projects must 
have staff that are sensitive to a wide range of human and environmental 
factors, and that can and will act upon those sensitivities as needed. 

The five categories of sensitivities or awarenesses most frequently cited 
as being needed in the 20 projects studied are displayed in Table 8.25. 

The five categories most frequently cited by project foci are displayed 
in Table 8.26. 

In many ways the sensitivity data reflect the same pattern, and demand 
the same observations, as the skill data just reported. The nature of the 
sensitivities cited, their distribution across project fcci, and the rela- 
tively small percentage of the total set of sensitivity statements accounted 
for by the five categories are all noteworthy. It is extremely inter- 
esting, for example, that the five most frequently cited sensitivity 
categories pertain to people related awareness. It is also interesting 
that the first three of the five categories pertain primarily to staff 
related sensitivities, the second two to consumer related sensitivities, 
and that none of the categories pertained directly to attitudes or person- 
ality characteristics. The implications for the design of personnel develop- 
ment programs and the conduct of projects are clear. 
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TABLE 8.25 



The Five Categories Used Most Frequently to Classify 
The Sensitivities Needed in Educational RDD&E 



Set UV 
Coding 


Primary Categories of 
Enaoling Sensitivities 


Project Focus 


Total 

(20) 


Res Dev Diff Eval 

(51_ (73 (53 L (33 


02 Capabilities and limitations 

16 Existing value systems 

03 Needs of self and others 

30 Response sets of tgt audiences 

22 Responses of target audiences 


10 39 16 11 

5 29 20 10 

2 21 23 4 

3 8 20 7 

7 12 9 3 


76 

64 

50 

38 

31 


Total number of sensitivity statements 
coded within the categories 


27 109 88 35 


259 


Total number of sensitivity statements 


81 264 217 123 


685 


Percentage of sensitivity statements 
coded with the five categories 


33% 41% 41% 28% 


38% 



The data reported in Table 8.25 assume a slightly different pattern 
in their distribution across RDD&E than did the skill data. There is, 
proportionately, a heavier loading of the five categories listed in develop- 
ment and diffusion projects than in research and evaluation projects, and 
the variability in use patterns between and within development and diffusion 
projects is considerably greater for sensitivities than skills. The per- 
centage of sensitivity statements that remain unaccounted for by the five 
categories, however, is approximately the same, (62% as opposed to 61%), 
and the number of categories of sensitivities into which the remaining 
statements are classified are as numerous (53 as opposed to 45). As a 
consequence, the reader also needs to study Appendix 7 to gain a sense of 
the full range and distribution of sensitivities identified within the 20 
projects studied. 

The data presented in Table 8.26 introduces six additional sensitivi- 
ties that are much needed or relied upon in the conduct of educational RDD&E 
projects. Within the context of research, a sensitivity to the values of one- 
self and of others and an awareness of various structural characteristics 
of the environment and/or field of operations are more often relied upon 
than the commonly shared sensitivity to the needs of self and others and 
sensitivity to the response sets of target audiences. Within the context 
of development, a s nsitivity to language barriers emerges as more often 
needed than an awareness of the response sets of targeted audiences. Within 
the context of diffusion a sensitivity to the interactions between oneself 
and others is relied upon more often than a sensitivity to the responses 
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TABLE 8.26 
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The Five Most Frequently Used Categories of Enabling Sensitivities, 
Their Frequency of Use, and the Percentage of Sensitivity 
Statements Coded Within Them 



Set UV 


Primary Categories of 
Enabling Sensitivities 


Freq 

of 

Use 


% of Sensi- 
tivity State- 
ments Coded 


Research Focus (5 Projects) 


02 Capabilities and limitations 

22 Responses of target audiences 

01 Values of self and others 

10 Awareness of structure 

16 Existing value systems 


10 13 

7 9 

6 7 

6 7 

5 6 


Totals (5 Categories) 


34 42% 


Development Focus (7 Projects) 


02 Capabilities and limitations 

16 Existing value systems 

03 Needs of self and others 

13 Language barriers 

22 Responses of target audiences 


39 15 

29 11 

21 8 

14 5 

12 5 


Totals (5 Categories) 


115 44% 


Diffusion Focus (5 Projects) 


03 Needs of self and others 

16 Existing value systems 

30 Response sets of target audiences 

02 Capabilities and limitations 

04 Interactions of self and others 


23 11 

20 9 

20 9 

16 7 

15 7 


Totals (5 Categories) 


94 43% 


Evaluation Focus (3 I -ejects) 


02 Capabilities and limitations 

16 Existing value systems 

47 Willingness to work as needed 

30 Response sets of target audience! 

21 Limitations of analyses/data 


11 9 

10 8 

9 7 

7 6 

6 5 


Totals (5 Categories) 


43 35% 
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of target audiences. Within the context of evaluation a sensitivity to 
the limitations of data analyses and a willingness to work as needed (a 
personality traic?) emerge as more often needed than sensitivity to the 
needs of self and others and to responses of target audiences. As would 
be expected from Table 8.25, sensitivity to the capabilities and limita- 
tions of self and others, and sensitivity to existing value systems, 
maintained their frequent use positions within all project foci. These 
were the only categories of the five categories listed in Table 8.25 to do 
so. As such, the data reported in Table 8.26 reinforce the need to 
become familiar with Appendix 9 for a full sense of the range and distri- 
bution of the data collected in the Oregon Studies on project related 
sensitivities . 

Table 8.27 contains the last display of work requirement data asso- 
ciated .:th the aggregate of project outputs. It contains a summary of the 
categories presented in Tables 8.25 and 8.26, and presents the percen- 
tage of sensitivity statements identified that have been coded within them. 



TABLE 8.27 



The Eleven Categories Used Most Frequently to Classify 
the Sensitivities Needed in Educational RDD&E, 
and the Percentage of Sensitivity Statements 
Coded Within Them by Project Focus 



Set UV 
Coding 


Primary Categories of 
Enabling Sensitivities 


Project Focus 


Total 


Res . 
(5) 


Dev. 

{?) 


Diff . 

(7) 


Eval. 

(3) 


(20) 


02 


Capabilities and limitations 


13% 


15% 


7% 


9% 


11% 


16 


Existing value systems 


6 


11 


9 


8 


9 


03 


Needs of self and others 


2 


8 


11 


3 


7 


30 


Response sets of tgt audiences 


4 


3 


9 


6 


6 


22 


Responses of target audiences 


9 


5 


4 


2 


5 


01 


Values of self and others 


7 


4 


5 


1 


4 


04 


Interactions of self & others 


4 


2 


7 


1 


4 


13 


Language barriers 


2 


5 


3 


2 


4 


21 


Limitations of analyses/data 


2 


2 


2 


5 . 


3 


10 


Awareness of structure 


7 


2 


2 


0 


2 


47 


Willingness to work as needed 


0 


1 


0 


7 


2 


Percentage of all skill statements 
included within these categories 


56% 


58% 


59% 


44% 


57% 
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Work Requirements Data In Perspective 



The linkage of the data presented in this chapter to data presented 
in Chapter 7 has already been discussed. When viewed separately from 
specific output relationships work requirements data have much the same 
information load as task data reported in relation tc jobs, that is the 
data describe the range and nature of work in educational RDD&E. The 
utility of such data rests in the degree to which it provides the trainer 
substantive information on which to design and conduct training in a 
broad sense. Thus the chapter Mas introduced the reader to the range and 
nature of the data when collected in relation to the outputs of educa- 
tional RDD&E and displayed in relation to the focus of each project 
within which they were found. 



Project Organized Data in Perspective 



The three preceding chapters have presented the data collected within 
the Oregon Studies from the point of view of its ran^e and its distribution 
by RDD&E. Chapter 6 reported data on the characteristics of project per- 
sonnel; Chapter 7 reported data on the nature of project outputs; and 
Chapter 8 reported data on the nature of work requirements within projects. 

In combination these three chapters were designed to meet the obligations 
assumed by the Oregon Studies to "map the parameters ‘ of the domain of 
educational RDD&E, and to analyze the data ured in that napping for simi- 
larities and differences across project foci within the domain. 

Aa the data were presented in Chapters 6, 7, and 8 the primary sampling 
unit, and thus the primary unit of analysis, was the project. So organized, 
the data have both strengths and weaknesses. Since the 20 projects studied 
were selected with a view to ,, mapping ,, the external parameters of the domain 
of educational RDD&E, they provide a reasonably good sampling base for por- 
traying the range of personnel, outputs, and work requirements found within 
the domain. Unfortunately, however, a sample of projects selected to 
establish the external parameters of a domain provides a relatively weak 
base for exploring similarities and differences within that domain. Accor- 
dingly, the data reported on similarities and differences across project 
foci can be taken only as suggestive or supportive of hypothesis generation. 
Under no conditions can they be construed as representing strong central 
tendency data. As a consequence the person who wishes to make use of the 
data reported thus far has in fact two sets of data to deal with, that 
reporting the range of things observed within the domain, and that reporting 
the central tendencies of things as they are distributed by RDD&E. While 
considerable confidence can be placed in the f irst, care must be taken against 
over-interpreting the second. In combination, however, the two sets of 
data should be of considerable value to persons responsible for operation of 
personnel development programs, for persons wishing to pursue further 
research on the nature of educational RDD&E, etc., for they represent the 
only data base of their kind that is available. 






Chapter 9 



OUTPUT DIMENSIONS AND INTERACTIONS 



The three previous chapters have organized the presentation of 
data around the projects. Data in those chapters was presented in an 
effort to map the domain of educational RDD&E using projects in the 
focal areas of research, development, diffusion and evaluation as 
organizers. The present chapter is the first of three chapters to 
analyze the data collected in the Oregon Studies where outputs instead 
of projects are used as the primary unit of analysis. In these chapters 
the concern with project sample that dominated the presentation of the 
data in the previous chapters is no longer a primary issue. In its 
place is the issue of the adequacy of the output sample.! The rationale 
for using outputs as a primary unit of analysis when looking at work 
requirements, for example, rests upon the simple notion that what 
needs to be produced in large part determines what needs to be done to 
produce it. It also rests upon the observation that outputs represent 
a much more stable unit of analysis for work requirements within RDD&E 
than organizers such as jobs or job roles. Outputs have a stability 
about them, both across and within projects, that allows work require- 
ments to be linked to them meaningfully and with a view to generali- 
zability. All the projects visited in the Oregon Studies and, assum- 
edly, all projects in the domain of educational RDD&E concern them- 
selves with the production of outputs of one kind or another. Because 
these outputs can be identified and subsequently analyzed regardless 
of the projects in which they are found and regardless of the focus or 
purpose of the project they represent a stable, consistent, and common 
element around which data can be gathered, analyzed and presented. 

The three chapters that report data using outputs as the unit of anal- 
ysis deal, respectively, with the dimensions of outputs, the work 
requirements associated with outputs, and a conceptual framework for 
viewing the relationship between outputs and the work required for 
their production. 



1 Obviously, since outputs are linked to projects, the matter of pro- 
ject and output sample are not independent. The point is, however, 
that when outputs are taken as the unit of analysis projects need to be 
treated only as the vehicles by which outputs are identified. For an 
analysis of the adequacy of the output sample within the Oregon Studies, 
see Chapter 8. 






An Orientation to the Output Data 



The focus of the present chapter is the dimensionality of outpuLs . 
it will be recalled from Chapter 7 that four dimensional properties of 
outputs were identified during the course of the Oregon Studies: (a.) the 

dimension of s tructure (products, events, conditions); (b) the dimension 
of function (policy setting, management, production); (c) the dimension 
of character (knowledge, technology, implementation, information); and 
(d) the dimension of level (focal, component, facili tat ive) . It will 
also be recalled that only the dimension of structure was dealt with 
in Chapter 7. The purpose of the present chapter is to present the data 
available on the remaining dimensions of outputs. 

Two observations need to be made by way of introduction to the data 
presented in the chapter. The first has to do with multiple classifica- 
tion. While the concept of multiple classification is not new to science 
or scientists, there is always an element of surprise, and usually relief, 
to discover that a particular phenomenon can be viewed from more than 
one perspective, and that the validity of the observation made tends to 
depend on the perspective taken. Such was the case when it was "dis- 
covered" that outputs of work effort had properties other than struc- 
ture, that is, properties other than being products or events or con- 
ditions. To find that outputs served particular functions within projects, 
that they had certain characteristics or substantive properties that 
identified them with research or development or diffusion or evaluation 
activities, that they were always nested within a hierarchically and 
interdependently ordered network of outputs, and that much of their 
meaning within the context of a particular project rested upon their 
placement within that network, provided a revealing new set of per- 
spectives. The four dimensions of outputs that have been labeled 
structure, function, character, and level accomnodate all of the prop- 
erties of outputs identified during the course of the Oregon Studies. ^ 

The second observation by way of orientation to the chapter is that 
the classifications given an output in terms of its function, character, 
and level are situation specific . Operationally this means, for pur- 
poses of classification, that a particular output plays a particular 
function within a particular project, and assumes a particular char- 
acter and holds a particular position or level within that project. 
Operationally, it also means that an identical output within another 
project could assume a totally different function, character, and 



2 It is not suggested, however, that the properties of dimensions iden- 
tified thus far are exhaustive of the dimensional properties of out- 
puts. Nor is it suggested that the properties identified thus far are 
even the most critical properties of outputs from the point of view of 
training or project operation. The fact that they have been identified, 
however, that they can be judged with a high degree of reliability, and 
that they seem to exhaust the properties of outputs that had to be under- 
stood in order to collect the kind of data sought within the Oregon 
Studies, establishes them as dimensions that need to be attended to with 
some seriousness . 



position. While the structure of outputs are stable from the point of 
view of classification, that is, a product or event or condition is 
always a product or event or condition, irrespective of its project 
context, the output dimensions of function, character, and level vary 
according to the project context within which they reside. The situa- 
tion specific nature of outputs can be seen best within the context of 
the output maps that have been prepared for each project studied. These 
are reported within the case profiles that appear as Volume IV of the 
series of volumes reporting the Oregon Studies. ^ 

Three analyses of output dimension data are presented in the 
chapter. The first deals with the properties or characteristics of 
each dimension individually. The purpose of this type of analysis is to 
convey the meaning of the various output dimensions identified, the 
sub-classifications given each dimension, and the distribution of the 
962 outputs identified in relation to each dimension and its sub-class- 
ifications. The second and third kinds of analyses deal with the 
interactions between dimensions . The second explores the relationship 
between the various two-way combinations of the three dimensions iden- 
tified, e.g,, the function played by outputs having differing charac- 
teris tics , and the third explores three-way combinations, e.g., the 
function played by outputs having differing characteristics within 
various project foci . Within all three kinds of analyses outputs are 
differentiated according to their structure, i.e., products, events, 
conditions . 



Definitions, Illustrations and Single 
Dimension Analyses 



Output Function 

The function of an output refers to the purpose it serves within 
the context of a particular project. Three broad functions were iden- 
tified within the Oregon Studies as a basis for data collection and 
analysis: the policy setting function, the management function, and 

the production function. The following definitions were adopted for 
these functions: 

Policy setting function - a classification given an output that 

establishes standards or guidelines 
for a proj ect . 



^ The recognition of the situation specific nature of outputs has major 
implications for persons wishing to use outputs as a basis for the 
organization of personnel development programs, as well as for persons 
interested in the further investigation of outputs empirically. It 
also has implications for understanding the data to be reported, for 
the power of the context in shaping the classification of some outputs 
was so great that understanding those classifications outside a pro- 
ject context is sometimes difficult. The reader is reminded that the 
case profiles provide that context when it is needed. 



Management function - a classification given an output that 

orchestrates the resources (time, per- 
sonnel, materials, space, information) 
available to a project for the realiza- 
tion of the outcomes expected from it; 
also a report of that orchestration. 

Production function - a classification given an output that 

is a part of the total fabrication 
effort of a project. 

Thus, any output of work effort within a project that established stand- 
ards or guidelines relative to the project was classified as serving 
a policy setting function. In like manner, any output of work effort 
that orchestrated the resources available to a project toward the 
realization of desired outcomes was classified as serving a management ^ 
function, and any output that contributed in direct fashion to the 
overall task of actually fabricating or assembling the outputs pro- 
duced within a project was classified as serving a production function. 

The data presented in Table 9.1 illustrate, at the cluster cate- 
gory level, the nature of the functions served by the ten most fre- 
quently appearing clusters of products, the five most frequently 
appearing clusters of events, and the five most frequently appearing 
clusters of conditions. Taking the first entry in the table as an 
example, the data in the table are to be interpreted as follows: of 

the 122 outputs identified as being as instance of a report or contract, 
79 served a management function, 43 served a production function, and 
none served a policy setting function. A complete listing of the cluster 
categories of products, events, and conditions by functions served 
appears as Appendix 10. A summary of the data appearing in Appendix 10 
is presented in Table 9.2. 

Three features of the data presented in Table 9.2 are noteworthy: 

(a) the large number of outputs that served the management function; 

(l?) the large proportion of events and conditions that served the manage- 
ment function; and (c) the small number of outputs that served the 
policy setting function. Needless to say, the sheer number of outputs 
identified as serving the management function came as a surprise. This 
is not to imply that the management function was unknown or unexpected, 
especially in light of the number of Managers , " project directors, 
etc. that were identified and interviewed within the projects studied 
(see Chapter 6). The relative amount of attention directed to the 
management function, however, was unexpected. It is possible, of 
course, due to the size of the projects studied or the imbalance in 
the sample in favor of development and diffusion projects, that the 
data reported could be inflated with respect to management. But even 
if it were, and corrections were made for it, the number of project 
outputs serving a raanagement function would still be impressive. On 
the basis of the data that are now available it is likely that too 
little attention has been paid the management function in the past, 
both within the field and in personnel development programs. 



TABLE 9.1 



The Functions Served by Selected Categories of Outputs 



Set DE 




Output Dimension: 


Function 


Output 


Coding 


Cluster Categories 


Policy 

Setting 


Mgt. 


Prod . 


Totals 



STRUCTURE: PRODUCTS 



01 


Reports /Contracts 


0 


79 


43 


122 


04 


Work Specif icat ions /Procedures 


3 


38 


37 


78 


27 


Data (including printouts) 


0 


38 


37 


75 


02 


Data Ins t rumen ts /Techniques 


0 


27 


47 


74 


08 


Plans /Designs /Theories /Mode Is 


7 


48 


18 


73 


05 


Work Management Systems 


0 


57 


5 ; 


62 


03 


Guides/Manuals 


3 


24 


29 


56 


10 


Publications 


1 


15 


36 


52 


07 


Resource Lists 


0 


17 


27 


44 


14 


Instructnl Units/Use w/Staff 


0 


4 


35 

1 

i 

1 


39 

I 

i 

1 



STRUCTURE : EVENTS 



32 


Recruiting/Placing Staff 


0 


21 


0 


21 


33 


Collecting/Analyzing Data 


0 


9 


7 


16 


52 


Adapting Materials 


0 


0 


12 


12 


43 


Establishing Opns Parameters 


7 


4 


0 


11 


17 


Workshops/Institutes 


0 


4 


6 


10 



STRUCTURE: CONDITIONS 



21 


Working Environment /Atmosphere 


0 


19 


0 


19 


20 


Cooperative Relationships 


1 


12 


1 


14 


34 


Coordinated Efforts 


1 


8 


1 


10 


22 


Duality Assurance 


1 


6 


0 


7 


23 


Accountability 


0 


6 


0 

i 

! 


6 



TABLE 9.2 



The Functions Served by Products, Events and Conditions 



Structure of 
Outputs 


Function of Outputs 


Ou tput 
Totals 


Policy-Setting 


Management j 


Products 


Product 


14 


361 


384 


759 


Event 


9 


89 


32 


130 


Condition 


3 


64 


6 


73 


TOTALS 


26 


514 


422 


962 



By and large, the proportion of events and conditions serving the 
management function was not surprising. Events and conditions typically 
serve a facilitative role, and management is typically thought of as 
f acilitat ive . The small number of products, events and conditions 
serving the policy setting function, however, was surprising. Given 
the definition of the policy function as ”es tablishing standards or 
guidelines for a project,” it seems obvious that policy setting must 
occur, but there was little evidence of its being formalized within the 
20 projects studied. Apparently, except in the case of very large or 
complex projects, policy setting occurs informally or "unconsciously” 
through the prerogatives of a project director or other influential 
people . 

Before leaving the function data the reader is encouraged to 
review Table 9.1 for a sense of the way in which the various categories 
of outputs are distributed by function. The variability by category is 
considerable, and this must be remembered in interpreting or acting 
upon the data presented in Table 9.2. Appendix 10 contains a complete 
listing of the output cluster categories and the functions served by 
the outputs contained therein. 



Output Character 

The character of an output refers to what it substantively repre- 
sents within the context of a particular project. For purposes of the 
Studies, four categories describing this substance were employed. These 
included an output as an instance of (a) knowledge , (b) a technology , 

(c) implementatio n, or (d) as an instance of information . For theo- 
retical reasons (see Chapter 4 in Volume III of the series of volumes 
reporting the Studies) the definitions given these characteristics are 
closely related to research, development, diffusion, and evaluation: 



Knowledge 



Technology 



- a classification given an output of research, 
i.e., an instance of established fact, princi- 
ple, law, etc., that is generalizab le and that 
can stand the test of empirical verification. 

- a classification given an output of development, 
i.e., an instance of a plan, procedure or pro- 
duct that, when applied, can bring about a 
desired end with a known degree of reliability. 



Implementation - a classification given an output of diffusion, 
i.e., an instance of the adoption and utiliza- 
tion of knowledge, information, and/or tech- 
nology . 

Information - a classification given an output of evaluation, 
i.e., an instance of reliable information about 
a given phenomenon within a context over which 
the user expects to exercise influence or about 
which he expects to make decisions . 

Thus defined, any output that both met the substantive criteria of 
knowledge, technology, etc., and was viewed or treated as such within 
the context of the project, was classified accordingly. 

The data presented in Table 9.3 illustrates the character ascribed 
to the ten most frequently appearing cluster categories of products, 
the five most frequently appearing cluster categories of events, and 
the five most frequently appearing cluster categories of conditions. 

The table is to be read in the same manner as was Table 9.1. A com- 
plete listing of the cluster categories of products, events, and con- 
ditions by the position held within the projects studied appear in 
Appendix 11. A summary of the data appearing in Appendix 11 is pre- 
sented in Table 9.4. 



From Table 9.4 it can be seen that more than half of all the out- 
puts identified were categorized as technology. More than 20 per cent 
of the outputs identified assumed the character of information and 
14 per cent assumed the character of implementation. Only 4 per cent 
of the 962 outputs identified assumed the character of knowledge. It 
is clear from these data that while any single project is likely to 
employ a wide range of outputs typically associated with research 
(knowledge), development (technology), diffusion (implementation), 
or evalxiation (information), by far the most used category into which 
outputs can be placed is technology. These data are in keeping with 
the hypothesis developed in the Schalock and Sell paper in Volume III 
of the Oregon Studies that holds that research, development, diffusion, 
and evaluation represent "nested sets" of problem solving activities. 



TABLE 9.3 



The Occurrence of Character Classifications Given Selected 

Categories of Outputs 







Output 


Dimension: Character 




Set DE 












Output 


Coding 


Cluster Categories 


Know 


Tech 


Imp 


Info 


Totals 



STRUCTURE: PRODUCTS 



01 


Reports /Con tracts 


10 


18 


16 


78 


122 


04 


Work Specifications/Procedures 


0 


74 


0 


4 


78 


27 


Data (including printouts) 


2 


3 


1 


69 


75 


02 


Data Instruments /Techniques 


0 


74 


0 


0 


74 


08 


Plans /Designs /Theories /Models 


3 


69 


1 


0 


73 


05 


Work Management Systems 


2 


47 


9 


4 


62 


03 


Guides /Manuals 


0 


55 


0 


1 


56 


10 


Publications 


0 


6 


36 


10 


52 


07 


Resource Lists 


0 


33 


1 


10 


44 


14 


Instructnl IJnits/Use w/Staff 


0 


39 


0 


0 


39 



STRUCTURE: EVENTS 



32 


Recruiting/Placing Staff 


2 


8 


8 


3 


21 


33 


Collecting/Analyzing Data 


1 


8 


0 


7 


16 


52 


Adapting Materials 


0 


12 


0 


0 


12 


43 


Establishing Opns Parameters 


0 


4 


2 


5 


11 


17 


Workshops/ Institutes 


0 


8 


2 


0 


10 



STRUCTURE: CONDITIONS 



21 


Working Environment /Atmosphere 


5 


5 


6 


3 


19 


20 


Cooperative Relationships 


2 


6 


5 


1 


14 


34 


Coordinated Efforts 


1 


6 


2 


1 


10 


22 


Quality Assurance 


1 


5 


1 


0 


7 


23 


Accountability 


2 


2 


1 


1 


6 



More is said about the interaction of project focus and character 
later in the chapter.^ 



TABLE 9.4 

The Character of Outputs by Structure 





Structure of 
Outputs 




Character 


of Output 




Output 

Totals 




Knowledge 


Technology 


Implementation 


Information 




Product 


17 


473 


85 


184 


759 


Event 


9 


70 


31 


20 


130 


Condition 


14 


33 


20 


6 

t 


73 


TOTALS 


40 


576 


136 


2.10 


962 



Output Level 

The level of an output refers to the position it holds with re- 
spect to all other outputs within the context of a particular project. 
Three categories were developed within the Oregon Studies to describe 
the various positions held by outputs: a focal position, a component 

position, and a facilitating position. The following definitions were 
adopted for these descriptors; 



^ It is recognized that the concept of output character, and the lan- 
guage used in relation to it, is difficult to manage. To help in this 
regard the reader is encouraged to refer to the definitions provided on 
the previous page, the GLOSSARY that accompanies the volume, the 
Schalock-Sell paper in Volume III of the series of volumes reporting 
the Studies, and the table in each profile that identifies the charac- 
ter classification given specific outputs. The reader needs to be 
warned, however, that the context specific procedures followed in 
character classifications makes the logic of some character codes as- 
signed to some outputs difficult to understand at face value. For a 
complete review of the definitions and the decision rules which de- 
scribe the context specific nature of the classification, refer to the 
operational definitions and decision rules contained in Volume V of the 
Series. Evidence presented in Chapter 4 of this volume attests to Liu; 
fact that coders were able to reliably apply the categories pertaining 
to output character in accordance with those decision rules. 



Focal output - an outcome of work effort expected by 

contractual obligation to emerge from 
a project. 

Component output - an outcome of work effort that consti- 
tutes an element of, or an approxima- 
tion to, a focal output. 

Facilitating output - an outcome of work effort that supports 

the generation of focal or component 
outputs, but is not in itself a part of 
such outputs . 

The reader is referred to the output maps that appear in each of the 
case profiles to see how these category sets are used to organize the 
interdependencies of the outputs identified within a project. Direc- 
tions for reading an output map are provided in the GUIDE to profile 
reading that accompanies the profiles in Volume IV of the series re- 
porting the Oregon Studies. 

The data presented in Table 9.5 illustrate the various positions 
held by the ten most frequently appearing cluster categories of prod- 
ucts, the five most frequently appearing cluster categories of events, 
and the five most frequently appearing cluster categories of con- 
ditions. A complete listing of the positions held by the cluster 
categories of products, events, and conditions identified within the 
projects studied appears in Appendix 12. A summary of the data appear- 
ing in Appendix 12 is presented in Table 9.6. 

It will be seen from the data presented in Table 9.6 that by 
far the greatest proportion of outputs served in a facilitative capa- 
city. In fact, approximately three of every four outputs were identi- 
fied as facilitating the production of other outputs. Put in another 
frame of reference, the data reported in Table 9.6 suggest that for 
every focal output undertaken an average of 4 facilitating outputs must 
be linked to it in some way. 

While the majority of events and conditions assume a facilitating 
role, it is interesting to note that 7 events and 8 conditions also 
assume focal position. Surprisingly, no conditions were identified 
at the component level. No explanation can be given for this, but its 
occurrence does raise some interesting questions about the nature cf 
conditions. Reference to the definition provided for conditions in 
Chapter 7 may clarify this phenomenon for the reader. 

It is also of interest to note from the data reported in Table 9.6 
that the average number of focal or contracted-for outputs identified 
for each project studied was 6.7 (134 focal products in 20 projects). 
While this figure has little meaning in and of itself ; the wide varia- 
tion in the size and duration of the projects studied does point up 
the fact that projects tend to have multiple contracted-for outputs. 
Some sense of the nature and variety of the outputs contracted for 
within the 20 projects studied can be gained ^rom Table 9.5. As can 
be seen from the data reported in the table, the majority of focal 



TABLE 9.5 



The Occurrence of Level Classifications Given Selected 
Categories of Outputs 



Set DE 
Coding 




Output Dimension: Level 


Output 

Totals 


Cluster Categories 


Focal 


Component 


Facil- 

itating 




STRUCTURE:- PRODUCTS 








01 


Reports /Contracts 


41 


18 


63 


122 


04 


Work Specifications/Procedures 


1 


19 


58 


78 


27 


Data (including printouts) 


6 


17 


52 


75 


02 


Data Ins truments /Techniques 


1 


26 


47 


74 


08 


Plans /Designs /Theories /Models 


7 


9 


57 


73 


05 


Work Management Systems 


5 


5 


52 


62 


03 


Guides /Manuals 


9 


17 


30 


56 


10 


Publications 


21 


13 


18 


52 


07 


Resource Lists 


7 


9 


57 


73 


14 


Instructnl Units/Use w/Staff 


1 


28 - 


10 


39 




STRUCTURE: EVENTS 










r 










32 


Recruiting/Placing Staff 


0 


0 


21 


21 


33 


Collecting/Analyzing Data 


0 


0 


16 


16 


52 


Adapting Materials 


0 


0 


12 


12 


43 


Establishing Opns Parameters 


3 


0 


8 


11 


17 


Workshops/ Institutes 


0 


4 


6 


10 




STRUCTURE: 


CONDITIONS 








21 


Working Environment/Atmosphere 


0 


0 


19 


19 


20 


Cooperative Relationships 


2 


0 


12 


14 


34 


Coordinated Efforts 


1 


0 


9 


10 


22 


Quality Assurance 


0 


0 


7 


7 


23 


Accountability 


0 


0 


6 


6 



outputs were products, and they tended to take the form of Reports/ 
Contracts, and Publications. If the categories of Guides/Manuals, 
Resource Lists, Guides /Designs /Theories /Models, and Work Management 
Systems are added to these two first categories, however, 90 of the 
105 instances of focal outputs identified assume the form of paper 
products! The full range of outputs contracted for by the 20 projects 
studied will be found in Appendix 12. 



TABLE 9.6 



The Occurrence of Level Classifications Given 
Products, Events and Conditions 



Structure 
of Outputs 


Level of Outputs 


Output 

Totals 


Focal 


Component 


Facilitating 


Product 


119 


212 


428 


759 


Event 


7 


5 


118 


a 30 


Condition 


8 


0 


65 


73 


TOTALS 


134 


217 


611 


962 



The data presented thus far in the chapter have shown how multiple 
classification systems can be brought to bear upon the analysis of out- 
puts, and how the 962 outputs are distributed within those classifica- 
tion systems. The next set of analyses deal with the interactions that 
occur between these classification systems. 



Two Dimensional Interactions 



Thus far in the chapter the dimensional properties of outputs 
that are free to vary across settings, that is, function, character 
and level, have been treated separately. Since these represent mul- 
tiple classifications of the same set of outputs, it is possible to 
examine the relationship between any two of them. It is possible, 
for example, to determine the character of the outputs that served a 
particular function, the functions served by outputs that held par- 
ticular positions in the hierarchical organization of outputs, etc. 

Four analyses are presented in the pages that follow. These 
involve the interactions between output function and character, out- 
put function and level, and output character and level. The rela- 
tionship between output character and project focus is also presented 
to provide the reader with a sense of the way outputs categorized by 
character a^e "nested" within each of the focal areas of the domain 
of educational RDD&E. Since the data presented in each case are in 
highly summarized form, and are easily read, a minimum of discussion 
will accompany their presentation. 



The Interaction of Output 
Function and Character 



Table 9.7 displays the relationship between the functional proper- 
ties of outputs and their character. 



TABLE 9.7 



The Interaction of Output Function and Character 



Function 
of Outputs 


Character of Outputs 


Output 

Totals 


Knowledge 


Technology 


Implementation 


Information 


Policy-Setting 


0 


19 


2 


5 


26 


Management 


26 


278 


80 


1.30 


514 


Production 


13 


279 


54 


76 


422 


TOTALS 


39 


576 


136 


211 


962 



The table can be read either horizontally or vertically. Reading 
horizontally the data show, for example, that in the 20 projects 
studied no outputs that were instances of knowledge occurred for 
purposes of policy setting, but that there were 19 instances of tech- 
nology, t~wo instances of implementation, and five of information that 
served a policy setting function. Reading vertically the data show, 
for example, that while no outputs meeting the criteria of knowledge 
served the purpose of policy setting, 26 that met such criteria 
served the p rpose of management and 13 served the purpose of produc- 
tion, No particular explanation can be offered for the patterning of 
the data reflected in the table, and no effort will be made to speak 
to its significance. The distribution of outputs that assumed the 
form of a technology, however, and the heavy demand of the management 
function on outputs of all kinds, are noteworthy. 



The Interaction of Output 
Function and Level 



Table 9.8 displays the relationship between the functional prop- 
erties of outputs and the position or level held in the hierarchical 
organization of outputs produced by a particular project. The table 
is to be read in the same va; as Table 9.7. Again, while no explana- 
tion will be offered for the patterning of the data that is reflected 
in the table, and no comment will be made as to the significance of 
its pattern, the distribution of focal outputs across functions, and 
the distribution of the various level classifications within each func- 
tion, are noteworthy. Especially striking in this regard is the find- 
ing that roughly 90 per cent of management related outputs were 



ITS 



161 



facilitating in nature, and that roughly 75 per cent of ail 
tating outputs supported the management function. Clearly, 
tirnship between the management and production functions is 
ar.d probably more interwoven than has been suspected in the 



facili- 
fhp rela- 
complex, 
pas t . 



TABLE 9.8 

The Interaction of Output Function and Level 



Function 
of Output 




of Outputs 




Output 

Totals 


Focal 


Component 


Facilitating 


Policy-Setting 


2 


i 


23 


26 


Management 


40 


15 


459 


514 


Production 


92 


201 


129 


422 


TOTALS 


134 


217 


611 


962 



The Interaction of Output 
Character and Level 



Table 9.9 displays the last of the possible interactions between 
output dimensions, namely, the relationship between output character 
and output level. 



TABLE 9.9 



The Interaction of Output Character and Level 



Level of 
Ou tputs 


Character of Outputs 


Output 

Totals 


Knowledge 


Technology 


Implementation 


Information 


Focal 


12 


56 


37 


29 


134 


Component 


3 


159 


23 


32 


217 


Facilitating 


25 


361 


76 


149 


611 


TOTALS 


40 


576 


136 


210 


962 



The table is to be read as the two previous tables were. The inter- 
pretations to be given the data that appear in the table, an"’ the use 
to be made of them, are left to the reader. 



The Interaction of Outpu t 
Character and Project Focus 



One othev data display that is of significance for theoretical 
reasons is that wnich shows the relationship between output character 
and project focus. It will be recalled that these two variables are 
closely related to one another conceptually in that the definitions 
of output character (knowledge, technology, implementation, information) 
oarallel the definitions of research, development, diffusion, and eval- 
uation. Put in reverse order, the outputs of research, development, 
diffusion, and evaluation are expected to be, by definitions adopted in 
the Oregon Studies, knowledge, technology, implementation, and informa- 
tion respectively. Given this parallelism, and the additional assump- 
tion that RDD&E represent an interdependent or '’nested set" of problem 
solving activities that always appear in concert in the solution of a 
particular educational problem (see the Schalock'-Se^l paper in Volume 
III of the Studies), two hypotheses about the relationship between out- 
put character and project focus can be formulated: (a) there should 

be a loading of knowledge outputs in research projects, technology out- 
puts in development projects, etc., 4 and (1b) each project focus should 
show the use of outputs that have the characteristics that reflect all 
other project foci. Taken together the two hypotheses would hold that 
while research projects, for example, should show a preponderance of 
the outputs associated with them to be knowledge outputs (the result 
of the definitional strategy), they should also show outputs that take 
the form of technology, information, and implementation (as predicted 
by the nested set hypothesis). The data displayed in Table 9.10 rep- 
resent a beginning test of these hypotheses. 5 

As will be seen from the data, both hypotheses are reasonably well 
supported. Knowledge outputs had by far their highest loading in 
research projects; outputs that assumed the form of a technology had 
their highest loading in development projects; and outputs that took 
the form of implementation had their highest loading in diffusion proj- 
ects. Also, with one exception, all project foci had outputs of all 
characteristics associated with them. As such, these data are seen as 
generally supportive of the two hypotheses. 



^ In order to interpret the data presented in Table 9. 1C most meaning- 
fully it needs to be understood that projects were classified in the 
sample as representing research, development, diffusion, or evaluation 
on the basis of the overriding intent or concern that characterized 
the project, not on the basis of a specific cont racted-f or output. For 
a detailed discussion of project selection and classification proce- 
dures, the reader is referred to Chapter III of the present volume for 
project selection procedures. 
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TABLE 9.10 



The Interaction of Output Character and Project Focus 



Project 

Focus 


Character of Output 


Total 


Knowledge 


Technology 


Implementation 


Information 


Research 


34 


58 


9 


18 


119 


Development 


2 


249 


15 


49 


315 


Diffusion 


4 


213 


101 


80 


398 


Evaluation 


0 


56 


11 


63 


130 


TOTALS 


40 


576 


136 


210 


962 



Three aspects of the data reported in Table 9.10, however, do not 
fully support these hypotheses. The high loading of technology outputs 
in diffusion projects; the failure of information outputs to load high- 
ly in evaluation projects; and the infrequent appearance of knowledge 
output*. in development, diffusion, and evaluation projects tend to 
indicate that the hypotheses presented may not be adequate. These con- 
ditions do not deny the viability of the hypotheses, however, and they 
can probably be accounted for in a number of ways. For example, it is 
possible to make the case that, by the very nature of educational RDD&E 
as evidenced by data presented previously, all projects have to rely 
heavily upon outputs that assume, the form of technology and informa- 
tion. If that is true, then the distributions observed for those par- 
ticular outputs in Table 9.10 would be expected. A reverse case could 
be built for the observed distribution of knowledge outputs: the con- 

ditions required to produce such outputs are either so demanding, or 
the criteria established for admitting an output to the status of 
knowledge so rigorous, that project foci otb than research can rarely 
afford the luxury of either producing or depending upon such outputs . 
Whatever the explanation may be, the data presented in the table offer 
a rich base for speculation and hypothesis generation. 



Three Dimensional Interactions 



The analyses just completed looked systematically at various two- 
way combinations of output dimensions. While interesting and sugges- 
tive, they lead quickly to the desire for further detail. The data on 
outputs in the Oregon Studies were collected and organized in such a 
way that more complex analyses of output dimensions are possible. 




Because these £re relatively costly analyses to produce, however, and 
because they require a large data base to avoid a surfeit of empty 
cells, only one example vill be presented of the kind ot three dimen- 
sional analyses possible with the data. 

The data presented in Table 9.11 show the relationsnip between 
output structure (products, events, conditions), project focus (R,D,D,E), 
and the output dimensions of function, character, and level. Each 
structure is represented by the clus ter categories most frequently 
cited for analyzed outputs. The rationale for using this particular 
data base for demonstration purposes was twofold: (a) the use of 

cluster categories, as opposed to primary categories, provides for as 
large a data base as possible; and (b) the use of analyzed outputs, 
as opposed to identified outputs, provides the reader an opportunity 
to see in greater detail the source of the ' ork requirement data re- 
ported in Chapters 8, 10, and 11. Fewer empty cells would have ap- 
peared in Table 9.11, of course, had the analysis included all iden- 
tified outputs. ^ The value to be gained by the strategy followed, 
however, seemed to outweigh the frustrations of the empty cells. 

The table can be read as follows. Using as a reference product 
cluster category 01, Reports /Contracts , and looking only at re- 
search projects, three of the 26 reports /contracts analyzed assumed 
the status of focal or cont rac ted-for outputs, none served as com- 
ponents of the contracted-f or outputs, and three served in a facili- 
tative capacity to the focal outputs. When analyzed in terms of the 
function served, five of six outputs served a management function 
and one served the purposes of production. When analyzed in terms of 
their character, two of the six outputs used were identified as being 
instances of knowledge, two as instances of information, and one each 
as instances of technology and implementation. Continuing within the 
same cluster category, and looking at development projects, one of 
the 26 reports /contracts analyzed assumed the status of a focal out- 
put, one the status of a component output, etc. The table can be 
read in the same way for any combination of output category, output 
dimension, and project focus. It is not possible to determine from 
the data presented in the table, which of the focal outputs served 
what function or assumed what character, but that information can be 
made available. 

Because the data are relatively self-exolanator once the struc- 
ture of the table is understood, more will not be said about them. 

The reader is encouraged to peruse the data presented in the table at 
some length, however, as some interesting patterns are reflected in it. 



0 While it is conceivable that a cluster category could be classified 
in any of the categories within the various dimensions, in a practical 
sense some such cla r fications would occur only rarely even in a broad 
sample. For example, one would not readily find an output having to 
do with the working environment of a project classified as f focal out- 
put (at least within the framework of current project generation and 
funding). 5 " 

irT* 



I6S 





CO 

3 

U 

0 

pH 

CO AJ 
AJ o 


c 

0 

•H 

CO 


Character 


O 

H- 

c 

M 


1 

Cl ro lA 

h h <r c> 


O O O O 

O O O O 

ro fs CM n 
rH 

rH o O O 


OOcnO 

o o o o 

CM CO CO rH 
rH 

O O O O 


o o o o 
o o o o 

vO N CM lA 

o o o o 


a. 

e 

M 


J3 

G 

G 

H 


rH CN a o 


0 

c 

>4 


CN O O O 


3 G 


c 














PH *r-» 


G 














AJ 0 


B 




TJ 










3 )-i 


*H 




0 










O PH 


Q 


c 


u 


rH O cA 


cm m o o 


O 00 >f o 


sr <r cm <r 


. 




0 


PH 










fA 


AJ 


*H 












0 J3 


3 


AJ 


AJ 


f 










CX, 


o 


60 


iA vO LO O 


CM O CM CM 


CM lA rH rH 


CN m O rH 


4-1 CO 


AJ 


c 


s 




rH 






0) c 


3 


3 












CO o 


O 


Pm 


rH 










•H 






0 


o o o o 


O CM o rH 


O O O O 


O O O O 


X) AJ 






PH 










G CO 
































G *H 
















G M 






G 










rH *H 






cO 


CO vj vj O 


CM ro n m 


CM is lA H 


m n n <r 


G CO 






Pm 




rH 






CO CO 
















CO 




rH 


a 










CO pH 




G 


B 










CJ 




> 


o 


O rH o CM 


o m o o 


O vO O O 


CM sj O H 


rH CH 




G 


cj 










H 0 G 




hP 












• CO 
















CA CO 0 






G 










c s: 

f rt n f-J 






0 
r T . 


CO rH lA m 


CM rH O O 


o o o o 


o o o o 


t-M U P 

t-4 *H 






^ ! 


i 








OQ AJ <M 
















<J CO 0 




AJ 












H a 




G 


CO 


CM rH 


M rH 


CM rH 


CM rH 


*H C 




G 


3 


CO > CM to 


CO > IM CO 


CO > CM CO 


CO > cm co 


CM 0 




**“> 


G 


G G *H > 


G G *H > 


G G -H > 


G G *H > 


CO AJ 




0 

1, 


0 
r* . 


04 Q Q W 


a a w 


«QQW 


04 Q Q W 


co 3 




M 

PH 


KM 










CO X) 
















rH ‘H 
















U M 
















AJ 
















rH CO 
















cO *H 
















C Q 
















0 
















•H G 
















CO J3 
















a aj 












CO 




G 










CO 


G 




B TJ 










rH 


H 




*H C 






CO 


/*s 


G 


3 ^ 


CO 


Q CO 






G 


vO 


XJ vO 


XJ rH 


G O 








•H 


CM 


O CM 


G CM 


3 CM 


G * 






U 






G 


cr 


J3 T3 


CO 


O i 


II 


\ II 


0 It 


•H U 


H G 


4J 


» 


60 




CO 


u 


C 


N 


CJ 


1 


G 


G 


G G 


PH G 


J3 G 




3 


1 


AJ 


CO 


■H CO 


\ CO 


G CO 


rH 


T3 


l 


CO 


3 


M 3 


CO 3 


G 3 


CO 


0 


* 


cj 




O 


C 


H 


c 


H 


i 




C0 M 


G UJ 


0 CM 


^ CM 




PH 




M 


AJ O 


J3 O 


*H O 


CO o 








G 


G 


CH 


AJ 


AJ 








AJ 


ctf Pa 


\ Pa 


CTJ >s 


c 




•• 




(0 


u o 


CO u 


G G 


G G 




G 




3 


AJ c 


C C 


mH c 


B C 




M 




rH 


(3 G 


60 G 


cm G 


3 G 




3 




U 


O 3 


■H 3 


•H 3 


U 3 




AJ 






cj cr 


co cr 


g cr 


aj a 4 




G 






\ G 


G G 


G G 


CO G 




3 






co u 


a u 


PH U 


p: m 




M 






AJ M 


\ <M 


CO M 


M C4-I 




AJ 






M ^ 


a w 


N-/ 






CO 






0 


c 




co 










a 


CO 


u 


AJ 




AJ 






G 


rH 


0 


CO 




3 






04 


PH 


3 


Q 




a 
















AJ 
















3 
















O 




W 60 
















tt c 
















*H 


rH 


00 




CM 








AJ X 


O 


O 


o 


o 








G 0 
















CO CJ 











■y'-V 







o o o o 



O O vO o 



o cn cm rH 



cn O M 



o o o o 



O i-H O O 



CM O O O 



O O iH O 



o o o o 
cm m oo m 
o o o o 



CN rH o CM 
O I"* H O 

o o o o 



CM m 00 H 



» — I r s '- O i-H 



o o o o 



' — I r — I O rH 




o o o o 



O O rH O 



CM H 

co > cm co 
G G *H > 
Q Q W 



M rH 
CO > <4-1 CO 
G G *H > 
0 i 5 P Q W 



<M 

*M II 
CO 

AJ 0) 
CO CO 
3 

60 

C CM 
*H O 
G 

CO Ps 
rH G 
PH c 
\ g 
60 3 
c cr 
•H G 
AJ U 
»H CM 



O 

0 ) 

o4 



CM 



CO 


rH 


AJ 




3 


11 


O 




M 


G 


c 


Oi 




3 


M 




Pu CAM 




0 


60 




C 


>s 


*H 


G 


no 


C 


3 


G 


»H 


3 


G 


cr 


c 


G 


•H 


M 


Vw/ 


CM 






CO 




AJ 




to 









CM 

CO 



CM 





an 



a\ 


CS 

a 


W 


■U 


a 


a 


PQ 


o 


< 


u 


H 


V-/ 



c 

0 

•H 

01 


Character 


o 

a 

c 

M 


o o a cn 




O O O CN 
O O a O 
O CN CN o 

a o cn o 


o o o o 
o o m o 
o m a o 
a o o o 


a 

a 

M 


o o o o 


a 

u 

oi 

H 


a o o o 


> 

O 

5 


o o o o 


C3 














01 














S 




XJ 










•H 


a 


o 










Q 


o 


u 


a o o a 




o o o o 


o o a o 




•H 


a 










4-J 


U 












a 


O 


4J 










Oh 


c 


00 


o o a a 




a CN O CN 


a m o 


4J 


a 


X 






a 




3 














o 




pH 














O 


o o o o 




o o o o 


o o a o 






a 








a 






u 














CTJ 


a o a cn 




a CN O CN 


-4 m <ro 






a 






a 






rH 


a 












01 


a 












> 


o 


o o o o 




o o o o 


o o o o 




01 


u 












a 
















u 














o 


o o o o 




o o o o 


O O CN O 






- J 


. 










4J 














O 


01 


a a 




a a 


a a 




01 


a 


oi > a o3 




oi > a o3 


oi > a o3 




•a 


o 


01 01 H p> 




oi oi a > 


oi oi a > 




O 


o 

a 


« Cl Q W 




03 Q Q W 


03 Q Q W 




04 








1 














1 

01 














u 














0> 














X ^-N 


/^N 








«p-\ 




am 


CN 








03 <T 




oi a 


oi a 






O 


u 




o 


cx 






00 


03 II 


01 


a ii 


a ii 




oi 


01 


Q 


C 


a 




4 


u 


4J 


01 


o 


< 01 


In oi 




c 


03 


00 01 


a 


\ 01 


C 01 




oi 


U 


e a 


a 


a o 


o a 




> 




♦H - 


•H 


G 


a 


w 


M 


n a 


T3 


oi a 


u a 






01 


^ o 


C 


a o 


03 O 






u 


a 


O 


c 


a 






CO 


03 ^ 


u 


o ^ 


01 ^ 




9 « 


a 


C o 




u u 


03 U 


oi 


a 


< C 




a G 


G 


1 


w 


u 


01 




> 01 


01 01 




a 




oo a 


• • 


a a 


> a 


J 


u 




c cr 


01 


w cr 


a cr 


u 




a oi 




01 


a oi 




a 




a m 


a 


00 M 


o3 M 


i 


u 




u a 


a 


C a 


u a 


4 


u 




<u w' 


<j 


a ^ 


01 N-X 






a 


a 




CX 








a 


a 


u 


o 


4 


j 




o 


a 


o 


o 




a 




u 


to 


3 


u 


i 


a, 












4 


j 






a 








a 


a oo 




H- 










Q c 




a 










•H 


a 


p 


a 


o 






4J TJ 


m 


5 


<N 


Cl 






01 o 














cn (j 












Output Dimension Data in Perspective 



Chapters 6, 7 and fi have briefly described the people within 
projects, identified the outputs produced, and presented the output 
work requirements found in projects. This Chapter has more fully 
described the nature of the outputs produced by projects. To this 
point, each set of data has been presented independently. The 
chapters which follow link these data sets together and begin to 
describe the manner in which the Oregon Studies data may be '’brought 
to life” for training purposes. 



Chapter 10 



OUTPUT-WORK REQUIREMENT INTERACTION 



At one level of analysis, the data to be reported in the present 
chapter has been reported previously. In Chapters 7 and 9, respectively, 
the outputs identified in the 20 projects studied, and the interactions 
between the dimensional properties of those outputs, were described. In 
Chapter 8 the work requirements associated with the production of outputs 
were described, though independently of the outputs to which those data 
were linked. In the present chapter these two data sets are brought to- 
gether, and work requirement data are reported in relation to the outputs 
that give those data full meaning. 

The rationale underlying the chapter is the same as that which led 
to the use of outputs as the organizing unit for work requirement data 
within the Oregon Studies. Outputs represent a tangible form of work 
effort; they provide a logical and defensible base for judging the con- 
sequences of work effort, i.e. , the quantity and quality of that which has 
been produced as a consequence of work; and they tend to be stable through- 
out the life of a project. As a consequence, work requirement data that 
is linked to the production of outputs has some hope of becoming generaliz- 
able. That is not the case when it is linked to less stable entities such 
as job roles or positions. 

Given such a point of view, outputs become not only a good vehicle 
for collecting work requirement data, but a useful vehicle for organizing 
and presenting it as well. If outputs do in fact crosscut project foci, 
and if the work required to produce a particular output to a particular 
standard is reasonably similar across project foci , then persons interested 
in the utilization of work requirement data should fina its organization 
around outputs extremely useful. This should be the caJe especially for 
the designers of personnel development programs and the directors of 
projects. 

The purpose of the present chapter is to link, in summary form, the 
work requirement data to specific classes of outputs. Operationally, 
this involves summarizing all the standards, tasks, and enablers identified 
in relation to the production of a particular class of output and present- 
ing them in conjunction with that class of output. The strategy can be 
illustrated as follows. In the course of the Oregon Studies, 26 outputs 
that fell within output Cluster Category 01, Reports/Contracts, were analyzed 
for their associated work requirements. Each of these 26 outputs had 
standard, task, and enabler data elicited for it. Because the 26 outputs 
shared the property of being a report or contract, two assumptions can be 



* The distinction has been made in the Oregon Studies between work require 
ment data and work activity data. Work requirement data relate to the 
production of project outputs, and work activity data relate to job role 
performance. The work activity data are reported in Chapter 12. 
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made: (a) the standards, tasks, and enablers data accumulated for the 26 

reports and contracts should share a great deal in common, at least if the 
01 category was not too broad in its definition; and (b) in combination, 
the work requirement data identified for the 26 outputs should be fairly 
exhaustive and representative of the production of reports and contracts 
within educational RDD&E generally. Work requirement data organized in 
such a manner should be extremely helpful for persons concerned with 
either the production of reports and contracts or the preparation of 
others to produce them. 

One consideration that must be kept in mind in organizing work require 
ment data around outputs is Lhe fact that the generalizability of such data 
and to some extent its utility, depends upon the range and number of out- 
puts analyzed within a particular category of output. Put in other terms, 
the utility or generalizability of output-work requirement data depends 
upon the adequacy of the sample of outputs on which it is based. Given 
that the focus of the Oregon Studies was on the identification of 

external parameters of projects and outputs rather than their central ten- 
dencies, the number of output categories identified that contain an exten- 
sive sample of outputs within them are few. Only eight cluster categories 
of products, for example, two cluster categories of events, and two cluster 
categories of conditions, have a frequency of ten or more outputs that 
have been coded and analyzed within them. As a consequence, from the 
point of view of its generalizability, the output-work requirement data for 
other cluster categories must be interpreted with reservation.^ 

For purposes of the present chapter, work requirement data have been 
summarized only fo . the five most frequently appearing cluster categories 
of products, the two most frequently appearing cluster categories of events 
and the two most frequently appearing cluster categories of conditions. 
Because of the extensiveness of these summaries they are presented in 
appendix form. Accordingly, Appendix 13, displays all of the standards 
held for the outputs analyzed within each of the 5 cluster categories for 
which work reauirement summaries have been prepared, Appendix 14 displays 
all of the tasks associated with the production of those same outputs, 
and Appendix 15 displays all of the enabling knowledges, skills, 



2 While confidence in the generalizability of output-work requirement data 
increases as the number of outputs on which it is based increases, it would 
seem that a good deal of useful information could be gained about work 
requirements within the domain of educational RDD&E from an analysis of 
categories of outputs having relatively few instances of outputs coded 
within them. The lower range in this regard is not clear, but utility 
could be found in an analysis of the work requirements associated with 
five or more outputs within a class of output, and it may extend to as 
few as three. If the latter criterion were adopted, useful output-work 
requirement s umm aries could be prepared for 28 of the 46 cluster categories 
of outputs analyzed within the Oregon Studies. If the criterion of fivr 
outputs per category of output were adopted, useful output-work requirement 
summaries could be prepared for 19 cluster categories. 



ana sensitivities drawn upon in their production. Output cluster cate- 
gories, because of the larger frequencies involved, were used in prepar- 
ing these summary analyses. It is possible, of course, to prepare similar 
analyses for primary categories of outputs, or for output cluster cate- 
gories that have a smaller number of outputs coded within them. 

Table 10.1 demonstrates the linkage that will be found in Appendices 
13, 14, and 15 between class of output and work requirement data. The 
data in Table 10.1 pertain only to standards, and only to one cluster cate- 
gory of outputs, but it is illustrative of the standards data presented 
for other cluster output categories, and illustrative of the nature of 
the data to be found for tasks and enablers as well. 

The table is to be road as follows. Twenty-six outputs that met the 
definition of output Cluster Category 01, Reports/Contracts, were analyzed 
for their associated standards. Eighty-three output standards and 16 
process standards were identified for these 26 reports and contracts. Of 
the 83 output standards identified, ten pertained to goal attainment, nine 
to completeness of content, etc. Of the 16 process standards identified 
four pertained to deadlines being met, two to performance being respected, 
and so on. By reading Table 10.1 in full it is possible to see both the 
nature of the standards held with respect to reports and contracts and 
the frequency with which various of those standards tend to be used. By 
reading Appendices 13, 14, and 15 in like manner, summary information on 
standards, tasks, and enablers can be gained for output cluster categories 
Reports/Contracts, Plans/Lesigns/Theories/Models , Work Specifications/ 
Procedures, Data, Work Management Systems, Recruiting/Placing Staff, 
Establishing Operational Parameters, Establishing a Work Environment/ 
Atmosphere, and Establishing Cooperative Relationships. 

By rer erring to Appendices 13, 14, and 15, one discovers that a 
wide range of standards, tasks, and enablers are associated with the 
production of reports and/or contracts. For the 26 outputs categorized 
as reports or contracts, 31 different categories of standards were applied, 
27 different categories of tasks were engaged in, and a combination of 
78 different categories of knowledges, skills, and sensitivities were 
relied upon in their production. In the same vein, a total of 99 state- 
ments of standards were identified in relation to the 26 outputs analyzed, 
351 statements of tasks were identified that were pursued in their 
production, and a total of 102 statements of knowledges, 140 statements 
of skills, and 73 statements of sensitivities were called upon in the 
course of their production. 

The frequency with which particular categories of standards, tasks, 
and enablers were employed in producing the 26 reports analyzed can 
also be determined. The standards most frequently applied required that 
the goals established for the output be attained, that the product be 
complete in its content, that it be accepted by others working on the 
project and by the projects sponsor, and that it communicate to its 
audience clearly. While other output standards were also applied, those 
mentioned give some idea of the nature of standards associated with the 

IfJS 
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TABLE 10.1 



An Illustration of the Linkage Between Class of Output 
and Class of Work Requirement Data 



Ceding Sets 


Output and Work Requirement Categories 


Freq . 


DE 

J 

LM 


Cluster Output Category 

Structure of Standards 

Primiry Category of Standard 


01 


Reports /Con tracts 


26 


1 


Output Standards 


83 


12 


Goal attainment 


10 


13 


Acceptance by others (in project) 


9 


01 


Completeness of content 


9 


04 


Communication and clarity 


7 


14 


Acceptance by sponsor 


7 


08 


Agreement/concurrence w/others 


5 


09 


Lack of errors/discrepancies 


5 


16 


Compares favorably 


4 


05 


Utility or value 


3 


07 


Personal satisfaction/feeling 


3 


11 


Appropriate design/content 


3 ^ 


15 


Compliance w/sponsor guideline 


3 


18 


Satisfactory appearance 


3 


24 


Terminology appropriate 


3 


17 


Internally consistent 


2 


19 


Logical criteria 


2 


21 


Sources of variance controlled 


2 


02 


Quantity of outputs/data 


1 


06 


Acceptance by users 


1 


29 


Meets design expectations 


1 


2 


Process Standards 


16 


04 


Deadlines are met 


4 


20 


Performance respected 


2 


34 


Impact of effort favorable 


1 2 


02 


Personnel are satisfied 


1 


11 


Cos ts consistent w/estimates 


1 


17 


External enthusiasm evident 


1 


24 


Costs acceptable for benefits 


1 


25 


Staff reflect trust 


1 


35 


Outputs distributed/reques ted 


1 


39 


Evidence of pre-planning 


1 


40 


Outputs published externally 


1 
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cluster cf outputs being considered. With respect to the standards held 
for the process of producing reports and contracts people in the field 
see it us important that deadlines be met, that performance be respected 
and that the impact of the effort involved in producing chese outputs 
be favorable. For any other output cluster category that might be chosen 
the standards applied, while they might carry some of the same labels, 
will occur v/i ih different frequencies and reflect different priorities. 

liven the standards associated with the outputs included under 
Reports/Contracts the next set of linkages that are meaningful relate 
to tasks. Of thr 351 tasks involved in producing the 26 reports and/ 
or contracts mentioned the ones most commonly engaged in center around 
Constructing and Assembling Components, Producing the Output itself, 
and Constructing and Using Measurement and Production Tools. The next 
most commonly pursued set of tasks rev jived around the collection and 
processing of data. These tasks inc.'ude the development administration, 
and scoring of data instruments and :.he interpretation of results. Other 
clusters of tasks included such efforts as the assessment of output 
quality, with an emphasis on proofreading and editing, effecting account- 
ability and quality control, designing the output, and clarj lying the 
problem addressed. As in the case of standards, other outputs may require 
the performance of similar sets of tasks, bur again, not in the same 
quantity or with the same set of priorities. 

Having considered the standards held for Reports/Contracts and hav- 
ing identified the tasxs involved in the production of those outputs to 
the standards held, the next level of linkage centers around the 
knowledges, skills, and sensitivities needed by project personnel to 
perform the tasks identified. The most commonly cited knowledges that 
a staff member must have pertain to the kinds of things that are taught 
in courses related to RDD&E, such as research design, and in courses 
covering technical anchor professional topics. The staff member also 
needs to know about the external variables surrounding the project in 
'vhich the output is to be produced. A variety of other knowledges, 
though less frequently cited, are also relied upon. These relate to 
things learned in formal schooling, things learned about the project 
oi its context, and things associated with the technical operation of 
a project. 

Along with the knowledges mentioned, staff members responsible for 
producing reports and/or contracts need to have a variety of skills and 
sensitivities. By far the most frequently relied upon skill is that of 
writing. Other skills include finding fits among elements and integrat- 
ing them, applying measurement tools, handling and analyzing data, and 
facilitating interaction between people. Sensitivities needed include 
an awareness of the interactions that occur between themselves and 
others, an awareness of the responses of the audience that the report 
or contract is targeted for, and an awareness of their own and others* 
needs. A wide range of other sensitivities is also called upon, but 
not as frequently as those mentioned. Again, these as well as the other 
sets of data can be found in Appendices 13, 14, and 15. 

As the reader scans these appendices he will see that each output 
cluster is linked to a relatively unique set of work requirements. The 
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standards, tasks, and enablers associated with the product cluster cate- 
gory Reports/Contracts, for example, differ nvite dramatically In both 
quantity and kind from those associated with th» event category Cooper- 
ative Relationships. It is these differences that demonstrate the power 
of outputs as a basis for organizing work requirement data. 

The data presented in Table 10.1 as well as Appendices 13, 14, and 
15, are reported in a highly summarized form. To further detail the 
distribution of standard, task, and enabler data, work requi: ement data 
have been organized for three cluster categories by the primary cate- 
gories that they encompass. These data appear as Appendices 16, 17, and 
18, Table 10,2 illustrates how the data are organized within those 
appendices and the linkage that exists between output and process 
standards ar.d tv; of the primary output categories that fall wi 1 f n 
the cluster category Reports/Contracts, From the table it can b< 'en 
that 11 output standards and one process standard were cited for tne 
two outputs identified by the primary category label Terminal Report. 

Fiv output standards and no process standards were cited for the three 
outputs identified by the primary category label Proposal. Further 
inspection of the table reveals that the two primary categories shown 
contain only five of the 26 outputs which fall within Cluster Category 
101, and in combination account for only J6 output standards and one 
process standard. Considering the bulkiness of such data, and the 
small number of outputs on which it is based, the reader can appreciate 
the decision to present such anlayses for illustrative purposes only. 

The data presented in Appendices 16, 17, and 18 are valuable data, how- 
ever, and they elaborate appreciably the data presented In Appendices 
13, 14, and 15. The reader is encouraged co attend to these appendices 
carefully, both as a source of useful i formation about the production 
of specific classes of outputs and as an illustration of the kind of 
data summaries that are possible with the Oregon Studies data. 



TABLE 10.2 



An Illustration of the Linkage Between Clas 4 ^ of Output and Class 
of Work Requirement Data, When Work Requirement Data are 
Ordered by Both Primary and Cluster Output Categories 



Coding Sets 


Output and Work Requirement Categories 


Freq. 


DE 

FGHI 

J 

LM 


Cluster Output Category 

Primary Output Category 
Structure of STandards 

Primary Category of Standard 


01 


Reports/Contracts 


26 


0006 


Terminal Report 


2 


1 


Output Standards 


11 


16 


Compares favorably 


3 


19 


Logical criteria 


2 


01 


Completeness of content 


1 


12 


Goal attainment 


1 


14 


Acceptance by sponsor 


1 


15 


Compliance w/sponsor guideline 


1 


18 


Satisfactory appearance 


1 


24 


Terminology appropriate 


1 


2 


Process Standards 


1 


20 


Performance respected 


1 


0007 


Proposal 


3 


1 


Output Standards 


5 


07 


Personal satisf action/f eeling 


1 


08 


Agreement/concurrence w/otiiers 


1 


09 


Lack of errors/discrepancies 


1 


14 


Acceptance by sponsor 


1 


17 


Internally consistent 


1 



Chapter 11 



OUTPUT TREES 



Chapter 11 is the second of the cvo chapters designed to illus- 
trate the linkage between outputs and work requirement data and as 
such, represents the culminating data chapter within the volume. The 
data reported in the chapter are only illustrative, however, so it is 
also the shortest chapter In tin* volume. 

It will be recalled that in the previous chapter classes of work 
requirement data were linked to classes of outputs. This provided, in 
the form of the appendices, separate, though exhaustive, summaries of 
the standards held for a particular class of output, the tasks engaged 
in to produce a particular class of output, and the knowledges, skills, 
and sensitivities relied upon to produce a particular class of output. 
It will also be recalled, however, (see Chapter 4, Fig. 4.4) that the 
model that guided the collection of output-work requirement data called 
for the linkage of all three classes of work requirementc to a partic- 
ular cJass of output simultaneously. This most complex linkage of out- 
put and work requirement data is referred to as an output t ree , and 
represents both conceptually and empirically the major contribution of 
the Oregon Studies. That figure has been reproduced below as Figure 
11.1, and that figure has been coupled with data as Figure 11.2. 




PIC. 11.1. Tt»a link* ga af output aad work raqulraoaot 
data that cooatltutaa aa output traa. 
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FIG. 11.2. 
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The data seta that appear within Figure 11.1 are by now familiar 
to the reader. Multiple primary categories are linked to output clus- 
ter categories, and multiple classes of standards, tasks, and enablers 
are linked to each primary category. The nat ire of the dato that re- 
side within these sets is also by now familiar. In combination , the 
data sets appearing in the figure are intended to represent the full 
array of work requirements associated with the production of a partic- 
ular class of output. This array is an example of an output tree. 

Figure 11.2 represents an application of the output tree concept 
displayed in Figure 11.1. In Figure 11.2 the output tree is focused 
on the primary output category Terminal Report, an instance of the out- 
put cluster category Reports /Contracts . Two terminal reports were 
analyzed during the course of the Oregon Studies and the work require- 
ment data displayed in the figure are the work requirements identified 
as being associated with their preparation. The categories of knowl- 
edges, skills, and sensitivities required of personnel to prepare the 
two reports, the categories of tasks performed in their preparation, 
and the categories of standards applied to the process of their prep- 
aration as well as the reports themselves are listed on the right side 
of the tree by frequency of occurrence. The number appearing after a 
category label indicates the number of related work activities identi- 
fied in relation to the production of the two reports. The data dis- 
played in Figure 1J.2 represent the sum total of work requirement data 
collected in relation to the two Terminal Reports studied. 

The reader needs to be aware that both the nature and number of 
the categories listed in Figure 11.2, as well as the number of related 
work activities Identified within each category, varies with the out- 
put cluster category chosen for analysis, the number of primary output 
categories linked to a particular cluster category, and the number of 
outputs classified within each primary output category. Thus, the 
work requirement data displayed in Figure 11.2 would take on a differ- 
ent character or pattern for another class of cluster category, or for 
classes of primary categories within the Reports /Contracts cluster. 

All classes of outputs analyzed, however, have an associated set of 
work requirement data, and in all cases these data can be displayed in 
the form of an output tree. 

This property permits output trees to be created at the cluster 
category level, at the primary category level, and even at the individ- 
ual output level. The creation of output trees at the cluster and pri- 
mary category levels requires the interrogation of the data bank devel- 
oped within the Oregon Studies. The creation of output trees at the 
individual output level requires the interrogation of the data reported 
in the case profiles. In the latter case it is also possible to iden- 
tify the initial work requirement descriptions provided by project per- 
sonnel interviewed that contribute the basis of all categorized data 
reported in the present volume. 

Without access to the data files, the reader can generate an out- 
put tree in one of two ways. The first way is to turn to Appendices 
13 through 18 and select, for a specific primary or cluster category 
of output, the work requirements associated with it. For the prepara- 
tion of output trees at the cluster category level use Appendices 13, 
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14, find 15. For the preparation of trees at the primary output level 
use \ppendices 16, 17, and IB. The latter would produce trees that 
would carry approximately the same amount of information that appears 
in Figure 11.2. The former would carry a much heavier load of infor- 
mation and would consequently require greater space for display. The 
procedures performed in preparing trees is, however, the same in both 
cases. Trees can be created for classes of outputs other than those 
that appear in Appendices 13 through 18, but their preparation requires 
recourse to the data files. 

The second way for a reader to produce an output tree Is to turn 
to Volume IV, select a specific profile, choose any interviewed out- 
put in that profile, and obtain for that output the associated work 
requirements. It will soon become obvious, however, that that is a 
time consuming process and one of value primarily for the "craftsman" 
who wishes to pursue the production of a specific output to a specific 
standard of excellence. The preparation of output treea on the basis 
of the data presented in Appendices 13 through 18 is a reasonably 
simple task, and one that provides a product that has a great deal of 
utility for a large number of people. 

In closing the discussion of output trees, it seems reasonable to 
caution the reader one more time to interpret all of the data presented 
in the Oregon Studies in light of the linkages that exist among data 
sets. Work requirement data, for example, in the absence of the out- 
puts to which they are linked, cannot be expected to convey the same 
information as they do when associated with outputs. The meaning of 
any bit of data reported in the Oregon Studies, be it c specific task, 
a knowledge, or an output, is highly dependent upon the other data to 
which it is linked. This is not to imply that data secs treated inde- 
pendently are without value, but it is to imply that such treatment 
has limited value and that in order to grasp their full value they 
need to be seen in context. 

Two formats for reporting data together with the context from 
which it was generated have emerged from the Studies, the format of a 
case profile and the format of an output tree. The former has been 
designed to accommodate all classes of data generated within the Stud- 
ies on an individual project basis, and the latter has been designed 
to accommodate output linked data on either an individual project or 
cross project basis. For optimal utility, the reader is encouraged to 
treat the data presented in the Oregon Studies within their appropriate 
contexts . 
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Chapter 1 2 



SPECIAL PURPOSE MAPS: SELECTED ANALYSIS 



This chapter varies in format from those data chapters which have 
preceded it. During the course of the Oregon Studies individuals have 
presented to the project staff, both privately and through the medium 
of three "Review and Refinement" conferences a number of "Special" 
issues of immediate concern to them. A number of these issues, although 
of extreme concern, were impossible to detail as they called for evalua- 
tive classes of data, e.g., is one management organizational pattern 
bette r than another? A number of the issues raised were focused upon 
classes of data which had not assumed a central focus within the method- 
ological procedures developed for the Oregon Studies. This chapter has 
as its focus, then, four issues which have been selected for further 
examination by the staff of the Oregon Studies. Selection was made based 
not only on the availability of data, but on the frequency with which 
the issues were raised, on the importance of the issues conceptually, 
and on the interest of the issues as judged by the staff of th<fc Studies. 

The first of the special purpose issues examined concerns the de- 
gree to which the variability of the output data is influenced by or 
attributable to the Context variables of Size and Setting. The Level, 
Function and Character of outputs are analyzed and discusred in refer- 
ence to these two variables. This issue is of special interest primarily 
because of the conceptual issues raised in the Schalock-Sell paper con- 
tained in Volume III of the series reporting the Oregon Studies. 

The second special issue examines the responses of project person- 
nel to the questionnaire utilized by the Oregon Studies in order to de- 
termine work activities by general "position" on projects. These data 
are treated utilizing factor analytic techniques in a unique attempt to 
organize the activities associated with various positions as a means of 
determining categories of Jobs or "Job Roles." By treating this issue 
it wai felt that the definition of job roles, utilizing the question- 
naire vlata, presented not only an interesting challenge but might also 
provide an initial set of factors which might serve as the basis for 
future work. (The reader may be particulary interested in comparing 
the 12 "Task Clusters" identified by the AERA task force investigation 
with those identified by the Oregon Studies in this regard. )1 



^See reference: Worthen, B. R. , Anderson, R. D. & Byers, Maureen L. 

A Study Of Selected F actors Related To The Training Of Researchers , Develop- 
ers, Diffusers, And Evaluators In Education * Final Report. Grant No. 
OFG-0-71-0617 (520) . November 1971. American Educational Research Associa- 
tion . 
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I lir f h i rd \ hhuv which (0 dealt with utilizes .is its org/inlzlng Mtruc- 
turt- the linkage of Output, Work Requirement and People data. The Initial 
(irpanl/er for discussion Is the concept of fob Role, Job role can In* 
seen to derive Its meaning from these classes of data. In addition, 
however, a number of other concepts are also included within the discus- 
sion, such as "management" and "organizational structure." By forcing 
on this issue an epportunity was perceived to recombine People linked 
data with Output and Work Requirement linked data . While this process 
has been alluded to within this volume, Special Issue III is the first 
attempt to utilize this kind of linkage. This issue also allowed the 
staff to examine the relationships between a set of Job role categories 
and a set of data which was collected without regard to job role. 

The fourth issue focuses upon other uses to which the Oregon Studies 
data can be put. The three issues preceding this one cannot begin to 
cover the variability of questions to which the Studies data could be 
applied. This issue, then, attempts to describe the 3 major data sets 
and the possibilities which exist for their recombination with respect 
to any potential set of specific questions. This fourth issue should 
provide for the reader some additional insight into recombinations of the 
data which are possible. 








Spec! al \ Hsue I : 



Context Variables : 



SI 7,0 and Setting 



To this point In Volume I people, outputs, and work requirements 
have been reviewed for the reader in terms of the range and frequency 
of categories by which the data is organized. The occurrences of people, 
output, and work requirement data have been further refined for the 
reader by organizing the data as it distributes itself by the context 
variable "Project Focus," i.e., research, development, etc. A number of 
other context variables were also utilized for classification purposes, 
including size and settings. 

Special Issue I will address the question of the degree to which 
variance in projects exists as a function of the context variables of 
size and setting. The full range of categories of fiscal year funding, 
one of the Oregon Studies* measures of size, are enumerated in the first 
column. These size categories have been combined in other chapters of 
this volume for convenience in discussion. The category labels are pre- 
sented in the middle column. The last column contains the number of 
projects falling within each category. 



20,000 or less 


- 


Small 


(3) 


20,001 - 100,000 


- 


Small 


(5) 


100,001 - 225,000 


- 


Medium 


(6) 


225,001 - 500,000 


- 


Large 


(3) 


500,001 or over 


- 


Large 


(3) 



The setting variable, as originally coded, utilized the eight cate- 
gories enumerated below. The numbers of sites appearing within each 
category are indicated parenthetically. 

1. educational research and development centers (2) 



2. regional education laboratories (3) 

3. colleges and/or universities (5) 

4. professional associations (1) 

5. private non-profit organizations (1) 

6. industry or commercial organizations (4) 

7. public schools and/or school districts (2) 

8. state departments of education. (1) 



As the reader can see by reviewing these categories, a variety of 
setting categories are presented, each of which includes a small number 
of sites. For purposes of data presentation and discussion, therefore, 
it was felt that a combining of categories was warranted. Categories 
1 and 2, 3 and 4, 5 and 6, and 7 and 8 were combined for this purpose. 
The end result of this process is a set of four categories. The Project 
Focus distribution of the sites visited by the Oregon Studies is sum- 
marized within Table 12.1 as ordered by both setting and size. 
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TABLE 12.1 



Tin 1 l)i b l r 1 but. I on of the Oregon Studies Project Sample by 
Project Size and Setting as Ordered Within Project Focus 



Dimensions of the Variables 



_ Pro jec t Focus 

Rese a rc h fpe velopnienTp)! f f u s 1 on f Sval ua t_lon_ 




Project 



Setting 



R&D Labs/Centers 

Colleges and/or Universities 

Private Organizations 

Public Schools or School Districts 



Table 12.1 should help the reader to understand the nature of the 
sample acquired as well as its distribution by Project Focus. 

It is fairly clear from Table 12.1 that the sample, as distributed 
by project focus, incorporates at least three of the four classes of 
settings for all but evaluation projects. In reviewing the size data 
from Table 12.1, it can be noted that the research projects tended to 
be in the medium to small end of the classification scale, the develop- 
ment projects tended to cluster toward the medium end, and the diffusion 
sites tended to be very large. The evaluation sites, on the other hand, 
represent the two extremes of the scale, two sites being classified as 
small and one site being classified as large. 

Table 12.2 orders the sites sampled by the Oregon Studies by just 
size and setting. 
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TABLE 12.2 



The 



Distribution of the Oregon Studies Project Sample 
Organized Within Project Size by Ptoject Setting 



as 



Project Size 


Project Setting 


R&D Labs/ 
Centers 


Colleges and/or 
Univer sit i cs 


Private 
Organizat ions 


Public Schools or 
School Districts 


20.000 or less 

20.001 to 100,000 

100.001 to 225,000 

225.001 lo 500,000 

500.001 or over 


11 1 

2 3 

12 1 2 

1 2 

1 2 



Table 12.2 Indicates that the smaller projects seem to be clustered 
within R6D labs and centers, and colleges and universities, while the 
very large projects tend to be dealt with by both private organizations 
and school districts* The medium size projects appear to be distributed 
fairly evenly across all categories. 

In attempting to view th'j additional context variables of size and 
setting, and any variations that might be attributed to them, the unit of 
analysis question once again presents itself. As should be obvious to 
the reader, any or all of the variables discussed in Chapters 6, 7, and 
8, and any or all of the dimensions of those variables could be organized 
and summarized in relation to project size and setting. In short all of 
the data from earlier chapters of this volume could be related to each 
of these context variables. Given a reasonable limitation on chapter 
length, however, a choice had to be made from among these variables. 

With Outputs serving both as the descriptor of what people are producing 
and the organizer for the Work Requirement data it was felt that examin- 
ing the Output data first might be most expedient in terms of reviewing 
other classes of data. Instead of presenting output data in the same 
degree of detail and explicitness as that presented within Chapter 7, 
however, the various classification dimensions utilized to describe 
outputs will serve as the variables of concern These include output 
level, function, and character. These three descriptors of outputs were 
dealt with within Chapter 9 of this volume in relation to Project Focus. 
The reader is, therefore, referred to Chapter 9 in order that he might 
compare the data presented on size and setting with that already pre- 
sented with respect to focus. 



o 



o 
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Tab le 
by project 



12.3 displays 
size. 



the dimensions of outputs as they distribute 



TABLE 12.3 

Project Size as Described by the Level, Function 
and Character of Identified Outputs 













1 








" "■■■■“ 1111 " 


Dimensions of the 










Project Size 










Variables 




20,000 


20, 


000 to 


100,001 to 


225 


,001 to 


500,001 




or 


less (3) a 


100, 


000 (5) 


225,000 (6) 


500 


,000 (3) 


or 


over (3) 


Output Level 


Focal 


ii 


(15%) b 


34 


(15%) 


25 (13%) 


20 


(14%) 


44 


(13%) 


Component 


12 


(16%) 


64 


(29%) 


29 (15%) 


40 


(27%) 


72 


(22%) 


Facilitating 


_51 


(69%) 


125 


(56%) 


134 (71%) 


88 


(59%) 


213 


(65%) 












Output Function 










Policy-Setting 


1 


( 1%) 


4 


( 2%) 


3 ( 2%) 


2 


( 1%) 


16 


( 5%) 


Management 


47 


(64%) 


101 


(45%) 


124 (66%) 


84 


(57%) 


159 


(48%) 


Production 


26 


(35%) 


118 


(53%) 


61 (33%) 


62 


(42%) 


155 


(47%) 


Output Character 


Knowledge 


25 


(34%) 


3 


( 1%) 


7 ( 4%) 


1 


( 1%) 


4 


( 1%) 


Technology 


30 


(41%) 


162 


(73%) 


119 (63%) 


93 


(63%) 


172 


(52%) 


Implementation 


5 


( 7%) 


6 


( 3%) 


23 (12%) 


19 


(13%) 


83 


(25%) 


Informat ion 


14 


(19%) 


52 


(2 3%) 


39 (21%) 


35 


(24%) 


70 


(21%) 



a The number of projects represented within each setting. 



k The number 11 in this column reports the exact number of outputs 

classified as focal within the category. For purposes of discussion 
these raw data have been adjusted for the different numbers of sites 
per category and converted to percentages. 



Table 12.3 provides for the reader two sets of data within each 
cell entry. From Table 12.3 the reader can see that for projects funded 
at a level of $20,000 or less, for exampxe, 11 focal outputs were iden- 
tified. The percentage figures shown were arrived at by calculating 

Z 03 
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tlie average number of focal outputs per project and dividing that number 
by the average number of outputs identified for each size category. By 
treating the data in this maimer, therefore, the various classifications 
of project size can be compared on an equal or similar basis. The reader 
can see that there is little variation in the percentage of outputs 
classified as focal, across size categories, and that both the component 
level and the facilitating level of outputs, while scattering a little 
more widely in terms of percentage, present no clear cut trend. 

In reviewing output function data it is fairly clear that the num-* 
her of outputs classified as policy setting begin to play a somewhat 
more prominent role within projects of $500,000 or more per fiscal year. 
It is also interesting to note that for those projects funded for over 
$500,000 the management and production functions divide fairly evenly 
in terms of output percentages. The split between management and pro- 
duction outputs for other sizes of projects does not appear to be as 
equal . 

In reviewing the data on output character, results which might 
not have been anticipated are in evidence. For instance, 3 4% of the 
outputs produced by projects of $20,000 or less per fiscal year are clas- 
sified as knowledge outputs, while 64% of the same set of outputs were 
classified as serving a management function. This knowledge categoriza- 
tion is consistent with the research focus of these projects. As was 
demonstrated throughout the earlier chapters of this volume, there is 
a heavy emphasis on technology outputs irrespective of project focus. 

The percentage of technology outputs across project size as displayed 
within Table 12.3 tend to verify this point. Percentage data on out- 
puts classified as implementation in character appear to suggest a trend. 
As the size of project increased the percentage of outputs classified 
as implementation also tended to increase. This is the only "trend" 
represented within the table. These data should be interpreted in light 
of the fact that the larger projects, for the most part, focus on devel- 
opment and diffusion (See Table 12.1). 

Reviewing the data in Table 12.3 vertically allows another level 
of analysis. Despite the project focus the reader can see from Table 
12.3 that the "balance" of outputs as classified by their character 
shifts as the dollar amounts shift. This could be interpreted as re- 
presenting a trend away from the production of some focal output to the 
use of whatever is being produced. Such review shows that for projects 
of $500,000 or more per fiscal year the character of outputs is some- 
what more diversified, 52% are classified as technology, 25% as imple- 
mentation, and 21% as ini. 'rmation . Once again the reader is directed 
to Table 12.1 as an aid in interpreting these results. 

In sum then, it would appear that the percentage of outputs clas- 
sified as focal, component, and facilitating remain fairly well distri- 
buted irrespective of project size. The function of outputs begins to 
show some differentiation across project size, with policy setting ac- 
counting for a greater percentage of outputs as size increases, and with 
the balance between management and production classifications of outputs 
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becoming more even as the project size increased. In reviewing output 
data classified by character a number of distinctions were elBo observed. 
The most Interesting trend represented by these data is the increased 
percentage of outputs classified as having a character of Implementation 
as the size of the projects increases. In reading Table 12.3 vertically 
a balancing of the character of outputs as project size increases be- 
comes evident. 



Setting 

Another interesting review of output data is provided through use 
of the context variable Setting. Table 12.4 organizes the output data 
on the basis of project setting in much the same way as did Table 12.3 
for project size. 



TABLE 12.4 

Project Setting as described by the Level, Function 
and Character of Identified Outputs 



Dimensions of the 
Viables 


Project Setting 


R&D Labs/ Colleges and/or Private Public Schools or 

Centers (5) a Universities (6) Organizations (5) School Districts (4 


Output Level 


Focal 

Component 

Facilitating 


20 (ll%) b 36 (16%) 49 (14%) 29 (14%) 

25 (14%) 67 (20%) 69 (20%) 55 (26%) 

136 (75%) 124 (55%) 224 (66%) 127 (60%) 


Output Function 


Policy-Setting 

Management 

Production 


4 ( 2%) 4 ( 2%) 14 ( 4%) 4 ( 2%) 

111 (61%) 116 (51%) 169 (49%) 118 (56%) 

67 (37%) 107 (47%) 159 (47%) 89 (42%) 


Output Character 


Knowledge 

Technology 

Implementation 

Information 


13 ( 7%) 18 (8%) 9 ( 3%) 0 ( 0%) 

105 (68%) 164 (72%) 181 (53%) 126 (60%) 

15 ( 8%) 15 ( 7%) 84 (25%) 22 (10%) 

49 (27%) 30 (13%) 68 (20%) 63 (30%) 



a The number of projects represented within each setting. 

b The number 20 reports the exact number of outputs classified as Focal within the 
R&D Lab/Center category. For purposes of discussion these raw data have been 
adjusted for the different numbers of sites per category and converted to per- 
centages . 



AvS can be seen from this table the percentage of outputs classed 
as focal docs not vary to any great degree across project setting, the 
range being from 11 to 16 percent. The percentages of outputs classed 
as component, however, do tend to range more widely given the different 
project settings, the low being 14%, the high being 30%. The greatest 
percentage of facilitating outputs is produced by the R&D lab/centers. 
Moving downward in the table to the Function of Output, it can be seen 
that this same project setting, R&D lab/centers, produced the largest 
percentage of outputs classified as management, 61%. It is also clear 
that the largest percentage of outputs classified as policy setting 
occurred with private organizations. It would also appear that private 
organizations give equal emphasis to outputs classed as management and 
outputs classed as production. Across all project settings, however, 
it is clear that outputs classed as management were always greater in 
percentage then those classed as production. The outputs classed as 
management are just as difficult to interpret using project setting as 
an organizer as they are using project size. A review of Table 12.2 
shows that while R&D labs ar\d centers have the highest percentage of 
management classified outputs, they do not include projects in the range 
of $500,000 or over per fiscal year. In fact the projects undertaken 
within the sample of R&D labs and centers tend to be on the medium to 
medium-low size. The private organizations, which appear to give equal 
emphasis to management and production classified outputs, include at 
least one project in the size range of over $500,000. The projects 
undertaken by private organizations within the sample can be classed 
as medium to medium-high in size. 

The next section of data contained within Table 12.4 deals with 
the output classification of character. As can be seen from the table, 
R&D Lab/Centers and Colleges and/or Universities would appear to have 
the greatest percentage of outputs classified as knowledge. Colleges 
and/or Universities also appear to have the highest percentage of out- 
puts which were classified as technology in character. The implementa- 
tion classification once again provides some interesting data. It would 
appear that private organizations represent the only category dealing 
with implementation to any significant extent. In looking back at Table 
12.2, it can be seen that the private organization data is obtained from 
a wide diversity of projects as described by size. From a review of 
Table 12.1 it is evident that the private organization data come from 
a research, a development, and three diffusion projects. The heavy em- 
phasis on diffusion in private organizations might explain the charac- 
ter data represented within Table 12.4. It is, however, somewhat more 
difficult to interpret the 10% figure for implementation within the 
Public Schools or School Districts classification. A review of Table 
12.1 shows that the public school sample included two development, one 
diffusion, and one evaluation project, all of which fa?.i within the 
medium to large category in terms of dollars. It would appear from 
these data that the projects undertaken within the public school or 
district setting are more concerned with the provision of information 
than with implementation. Again this is somewhat difficult to inter- 
pret given the distribution of these projects as noted within Table 
12.1 and 12.2. 
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In summary, it might be pointed out that colleges and universities 
within this sample were primarily concerned with the production techno- 
logy outputs, as were the R&D labs and centers. Private organizations, 
on the other hand, appeared to be more concerned with implementation re- 
lated outputs, while public schools and school districts were apparently 
more concerned with the generation of information. Interesting as these 
data might be the reader must keep in mind that until a much larger sample 
rounds out the cell entries within Tables 12.1 and 12.2, the prediction or 
the identification of the causes of such relationships as are dipicted 
within Tables 12.3 and 12.4 remains quite speculative. 
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Special Issue II: Work Roles of RDD&E Personnel 



The many problems associated with specifying "job role" and the 
variations which were associated with it, led the Oregon Studies away 
from the utilization of this concept as a prime organizer of data. Its 
utility as a convinient organizer of personnel, however, kept the ques- 
tion of definition open during the course of the Oregon Studies. It 
was felt that if such vaiiables as management style, organizational pat- 
tern, etc. could be removed as contributing sources of variation in the 
definition of job role, there might be some means of coming back to the 
use of the concept as an organizer of data. 

Responses on the significance of several categories of activities 
were obtained from 90 project personnel through the use of Form 04, the 
"General Position Activities Questionnaire." Availability of this data, 
provided by use of the 8-interval Hemphill scale of item significance, 
made possible an opportunity to explore the issue of how these respond- 
ents tended to cluster in terms of their responses to each activity. 

If significant clustering of people could be identified, a start would 
be provided toward the empirical definition of job roles within educa- 
tional RDD&E. 

At the suggestion of Dr. Paul D. Hood, I a factor analysis of Form 
04 responses was attempted. An intercorrelation matrix based on 87 
usable responses was generated to form the basis of a Q-type factor 
analysis. Rather than factoring the activity items, this approach would 
factor the people who responded to them. Interest was in defining the 
significant characteristics of job roles of personnel, not of interre- 
lated or covarying job tasks. 

The Q-type analysis was performed using the BIMED General Factor 
Analysis program, BMD03M, on the CDC 3300 computer at the Oregon State 
University Computer Center. This program included computer rotation of 
the resultant factor matrix. 

Since this analysis effort was only exploratory, only the highlights 
of the factor results are presented in this volume. If the reader wishes, 
he may contact Teaching Research for copies of the computed intercor- 
relation matrix, eigenvalues, eigenvectors, unrotated factor matrix, 
and rotated factor matrix. The factor matrix contained 23 factors, ac- 
counting for 86 percent of the total variance. For exploratory analysis 
purposes, only the first ten factors were examined tor definitioral im- 
plications. These factors are listed below: 



Factor 1: Staff Worker (low-level data analysis 

and instrument development) 

Factor 2: Staff Worker (high-level statistical 

design, analysis and instrument design) 



^Personal communication, March 1971. 
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Factor 



3: 



Supervisor (high-level data analysis) 



Fact, or 




Manager (mid-management , 
personnel contact) 


with staff 


Factor 


5: 


Management Representative 


(outside contacts) 


Factor 


6: 


Staff Worker (literature 


abstractor) 


Factor 


7: 


Undefined (possibly "team 


leader") 


Factor 


8: 


Information Disseminator 




Factor 


9: 


Undefined (possibly "research aide") 


Factor 


10: 


Undefined (possibly "new 
employee with statistical 


doctorate- level 
skills") 



The ten factors listed accounted for 62% of the total variance. 
Seventy five of the 87 persons were represented with factor loadings 
°f + .30 or greater. Factor 1 accounted for 18% of the variance. 
Appendix 19 contains the factor loading of each person who loaded signi- 
ficantly on each factor. Personnel background information used in col- 
lating this data was based on Items 5, (sex) 7, (highest degree) 46, 
(relationships to primary contractual project) and 47 (coding Set X) of 
questionnaire Form 03 (see Appendix 2, Volume V). In addition, Appendix 
20 summarizes the nine General Categories of work and the Specific 
Categories of work by Project Focus, Project Level and "Role." 

A brief summary of these ten factors is given below. This summary 
includes a factor label (where reasonable to do so) , a narrative des- 
cription of each job role represented by these factors, and the top five 
positive and negative discriminating items. As may le noted in the 
factor labeling and narrative description, three of the ten factors do 
not lend themselves to assurarce of a job role definition. The other 
seven factors, however, appeared reasonably certain of role character- 
istics, including roles for the production staff as well as for various 
levels and types of managers within projects. 



Factor 1; Staff Worker (low-level data analysis and instrument development) 

This factor is characterized primarily by performance of activities 
involving the analysis of data, including (in descending order of factor 
differentiation) such features as measures of central tendency, frequency 
tallies, correlation coefficients, item analysis, simple analysis of 
variance, simple tests of significance of observed differences, and non- 
quantified information. It includes some development of test items aud 
construction of questionnaires. These factor activities are distinguished 
from supervisory functions, which are clustered at the opposite polar end 
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of this factor. The factor appears to represent a job role in which 
females outnumbered males, the bachelor degree tends to be the highest 
academic level, participants are regular members of the project staff, 
and the position generally does not occur in diffusion-focused projects. 



Factor 2: Staff Worker (high-level statistical design, analysis, and 

instrument design) 

This factor is characterized primarily by performance of activities 
involving the design and analysis of relatively sophisticated statistical 
treatments. Design includes issues of sampling, designating the general 
statistical treatments to be used, and computer system models for the 
data. Data analysis includes such features as analysis of variance, 
simple tallies, non-quant if ied information, regression analysis, and 
item analyses. Accompanying the statistical design is substantial in- 
volvement in developing observational techniques, as well as some acti- 
vity in constructing questionnaires, test items, and interview outlines 
ana schedules. These factor activities are distinguished from project 
management functions, which are clustered at the opposite polar end of 
this factor (particularly involving writing and coordinating activities). 
The factor role occurred only in evaluation and development projects, 
and individuals with highest factor loadings were not regular members 
of the project but were lending their expertise from within the total 
agency . 



Factor 3: Supervisor (high-level data analysis) 

This factor is characterized primarily by performance of many of 
the same statistical design and analysis activities of Factor 2, but 
also including the element of staff supervision activities. Design 
includes issues of computer system models for the data and designating 
the general statistical treatments to be used. Data analysis includes 
such features as correlations, simple tests of significance of observed 
differences, analysis of variance, measures of central tendency, regres- 
sion analysis, frequency tallies, and item analyses. Supervision includes 
the allocation of responsibilities to project personnel and review of 
staff performance. These factor activities are distinguished from those 
involving the tryout of new curriculum developments at the level of class- 
room contact which are clustered at the opposite polar end of this 
factor. The factor appears to represent a job role involved primarily in 
research or evaluation projects, rather than in development or diffusion 
projects. 



Factor 4: Manager (mid-management, with staff personnel contact) 



This factor is characterized primarily by performance of management 
activities involving direct personal contact with staff and others. 
Writing of research proposals and reading of inhouse materials are of 
high significance. Staff procurement and development are the concern, 
as well as a variety of project coordinating functions. These factor 
activities are distinguished from a project director’s managerial func- 
tions involving project conceptual design and reporting, which are 
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clustered at the opposite polar end of this factor. The factor appears 
to represent a middle-management job role involved primarily in develop- 
ment projects, but not generally in research and evaluation projects. 
Consistent with the mid-management tendency of the factor, personnel 
generally are males with master’s degrees, not with doctorate degrees. 



Factor 5: Management Representative (outside contacts) 

This factor is characterized primarily by performance of activities 
involving contacts outside of the project itself. Thus, contacts with 
higher agency management, funding sponsor, and visiting personnel are 
involved, as are presentations made at professional meetings. These 
factor activities are distinguished from those involving project data 
collection, development of data collection instruments, and designing/ 
planning of procedural activities for the project. The factor appears 
to represent a job role involved in development or diffusion projects, 
but not generally in research or evaluation, though the personal pres- 
tige of the individual and of his efforts in other projects may cause 
some apparent deviation from this pattern. Both master and doctorate 
levels of academic attainment are represented. 



Factor 6: Staff Worker (literature abstractor) 

This factor is characterized primarily by performance of reading 
activities. While obviously occurring at the ERIC facility, three other 
development or diffusion projects also contained this job role. Scholarly 
essays, inhouse material, general literature surveying, and methodologi- 
cal documents were the subjects of this reading function. These factor 
activities are distinguished from those involving the design and develop- 
ment of empirical studies. The factor, at both polar extremes, appears 
to represent a job role not involved in research projects, though no 
research projects were examined in their early formative stage. Females 
tend to predominate in this job role. 



factor 7: Undefined, but possibly "team leader” 

This factor is characterized primarily by performance of what might 
be termed ”team leader" activities. Clustering of personnel character- 
istics and of activity significance is meager, making interpretation of 
this factor uncertain. With only one person loading on the negative side 
of the factor, contrasts are potentially unreliable. However, predomin- 
ant positive activities did include the scheduling of project activities 
and the allocation of responsibilities to project personnel, as well as 
some degree of designing and planning of project procedures. Writing 
tasks involving correspondence and interim, status, or periodic reports 
were rated as highly significant. Collecting and analyzing data were 
not a major part of the factor, nor was the development of research 
tools or other information-gathering instruments. Diffusion projects 
were the dominant vehicle for this job role. 
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Factor 8: Information Disseminator 



This factor is characterized primarily by performance of communica- 
tive activities, both oral and wTitten. However, with only two persons 
loading on the positive side of this factor, and their factor loadings 
not exceeding .36, interpretation of this factor is somewhat hesitant. 
The predominance of communicative tasks, nevertheless, does tend to 
support the factor definition as being highly related to information 
diffusion job functions, though a wide variety of other types of acti- 
vities also appear as part of the job. Particularly significant are 
teaching functions in classrooms, seminars or workshops, and profes- 
sional meetings. Generally of a non-quant itative nature, the job acti- 
vities tend to occur in a limited fashion in diffusion or development 
projects, but not in evaluation or research projects. 



Factor 9: Undefined, but possibly "research aide" 

This factor is characterized primarily by a seeming lack of in- 
volvement in the project efforts and mission, with the reading of re- 
cent project-related research, methodological documents, inhouse mater- 
ials, and scholarly essays as the predominant activities. Some activity 
in developing project methodologies and in establishing contact with 
other personnel or agencies is considered relevant, but all other speci- 
fic activities were rated of low significance. Supervision and teaching 
were generally not part of the job. The lack of individuals loading 
positively on this factor makes firm definition of the job role uncertain, 
however. Persons at the opposite polar extreme of this factor were of 
the order of project directors and managers, heavily involved in the 
project efforts. Neither research nor evaluation projects were involved 
in either polar extreme of this factor. 



Factor 10: Undefined, but possibly "new doctorate-level employee with 

statistical skills'* 

This factor is characterized primarily by a concentration of perfor- 
mance on high-level data collection and statistical analysis, as opposed 
to top-level management of several projects within a large R&D agency. 
Procedural design functions are a strong part of the job, but most other 
features are of relatively low significance. Again, the sparsity of 
persons entering into the makeup of this factor prevents serious con- 
trasts and factor definition. However, both persons with high negative 
loadings on the factor were well-established Principal Investigators. 

All persons held the doctorate degree. The positive side tends to ap- 
pear like a recent graduate being used purely in the realm of empirical 
investigation, with little writing or supervising assignments as yet. 



Obviously, these factors need further examination before firm con- 
clusions may be drawn about their meaning. It is of interest to note 
the extent to which the factors have meaning on the basis of personal know- 
ledge of each person represented. Interviewers from the Oregon Studies 



197 



- . • -o 





f 



¥ 






i 

\ 

i 



i 



f 



examined each factor and the personnel they represented. Clustered job 
roles were In their judgment very apparent and real. Further research, 
using more project personnel as well as an improved listing of work 
activities, would appear to be desirable. A significant improvement 
in the Form 04 listing can readily be provided by adding the 21 items 
from Form 04-A to those used, and by examining the j 722 task statements 
and the 280 primary task categories that were generated by the Oregon 
Studies . 

The exploratory nature of this effort to cluster job roles is em- 
phasized to prevent the reader from making unwarranted conclusions on 
the basis of these results. Activity statements were conceived prior 
to going out onsite and gathering actual performance statements. Thus, 
the activity statements on questionnaire Form 04 tend to overemphasize 
the traditional professional activities at the expense of diffusion and 
development activities, and at the expense of those functions performed 
by persons trained only to the bachelors degree level. The ability to 
specify statistical efforts in some detail also allowed such items to 
assume more power in differentiating job roles than may be the actual 
case. Now that specific task statements are available (through the 
overall effort of the Oregon Studies and other complementary projects 
sponsored by USOE) there is a firm basis for revising the inventory of 
RDD&E activities to yield a more comprehensive and specific set of j terns 
on which to differentiate job roles# 
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Special Issue III: Job Roles - Outputs and Work Requirements 



Quite early in the course of the Oregon Studies it was discovered 
that similar job role categories were invariably defined differently 
by the various projects visited. It was not uncommon to find equivalent 
job role titles across projects and, at the same time, to find different 
activities being engaged in by the individuals holding these similar 
titles. Job role titles varied to such an extent that it became impos- 
sible to deal logically with the resultant data. In short, the concept 
was expected to provide a discriminative power which would enable data 
to be accurately categorized, but early data from the Studies indicated 
that such was not the case. 

The issue discussed here will address the relationship between 
job roles as coded on the Form 03 questionnaire and the output and work 
requirement data collected independently from job role. The use of job 
role, as described in the previous section of this chapter, provides 
the reader with some commonly used organizers as pivotal points from 
which to review the output and work requirement data. It also provides 
an opportunity for the reader to judge for himself the advisability 
of abandoning or reconceptualizing job role as an organizer of data. 

It also provides the opportunity to discuss management and organizational 
structures which are based upon job role types of categories. 

As an integral part of the interview process of the Oregon Studies 
each interviewee was classified as to his job role. Eight categories 
of job role were utilized for purposes of classification. These are 
listed below with the number of individual interviewees classified with- 
in each category appearing in parentheses. The classification category 
of an interviewee, although initially suggested by the interviewer from 
the Oregon Studies, was verified with the interviewee. If a discrepancy 
in role definition existed, it was mutually resolved and the coding was 
modified accordingly. 



"High Level Managers 11 constitute the first three categories of job role 
and "Low Level Managers" constitute the next two categories when the 
data on job roles is clustered for purposes of analysis or interpreta- 
tion. The last three categories listed above are excluded from discus- 
sion in Issue III. 



Clerical Support Staff 
Technical Support Staff 
Advisor or Consultant 



Principal Investigator 
Project Director 



Top Level Project Manager 
Firs t- line Supervisor 
Professional Staff Member 



(14) 
( 12 ) 
(25) 
( 5) 
(48) 
( 1 ) 
( A) 
( 6 ) 
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Tabli* 1 2 . *3 pruHiuitH output data classified by Hiruclure hh dln- 
trlbuted by Job roli\s. Ah with earlier discussions of data, the raw 
data and the data indicating percentages (which have been adjusted 
for varying numbers of individuals) are presented within the table. 

From the table it can be seen that less than 50% of the outputs of indi- 
viduals classified within the first two categories of Job role were 
coded as products* The remainder of the outputs of the first two classes 
were fairly well distributed between events and conditions. The first 
two categories of individuals are, then, apparently primarily concerned 
with "facilitating" in the roles that they play. They are responsible 
for establishing and maintaining a series of events and conditions which 
promote and insure that the work associated with the contracted for 
outputs of the project are successfully accomplished. Events and condi- 
tions data refer to such responsibilities as "staff morale," "staff meet- 
ings , " etc , 



Table 12.5 The Structure of Outputs as ordered by Job Role 



Job 

Role _ 


Output Structure 


Product 


Event 


Condition 


Principal Investigator (14) a 


20 (48%)® 


10 (24%) 


12 (29%) 


Project Director (12) 


23 (50%) 


10 (22%) 


13 (28%) 


Top Level Project Manager (25) 


40 (73%) 


5 ( 9%) 


10 (18%) 


First Line Supervisor (5) 


7 (70%) 


1 (10%) 


2 (20%) 


Professional Staff Member (48) 


80 (81%) 


11 (12%) 


2 ( 2%) 


Clerical Support Staff (1) 


1 (100%; 


- - 


- - 


Technical Support Staff (4) 


7 (78%) 


2 (22%) 


- - 


Advisor or Consultant (6) 


3 (60%) 


1 (21%) 


1 (21%) 



a The number of individuals classified within the category. 

b The number 20 represents the exact number of Outputs classified as Product 
within the Principal Investigator, For purposes of discussion these raw data 
have been transferred to percentage in order to account for the different num- 
bers of individuals within each category. 



For the next two categories of job role, i.e., top level project 
manager and first line supervisor, approximately 25% more outputs are 
coded as products than for the first two categories of job roles. 

While personnel in these two categories of job role also generate a 
substantial number of conditions, it would appear that they are respon- 
sible for significantly fewer events than were the first two categories 
of job role mentioned. Personnel in this second clustering of job role 
are more responsible for producing product outputs more than they are 
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with the establishment of events. Personnel falling within the job 
role classification "professional staff member" clearly devote the 
majority of their energy to outputs classified as products. The con- 
l cern with conditions in this category is minimal, as is the concern 

, with events. As might be expected, clerical support etaff were totally 

product oriented. The technical support staff, while primarily respon- 
I sible for outputs classified as products, also had 22% of their outputs 

P devoted to staging events. The advisor or consultant Job role category 

t is very similar in percentage distribution to the principal investiga- 

I tor and project director categories. This would give some indication 

of the nature of the advising or consultation help derived from such 
individuals. 

The data represented in Table 12.5 indicate a number of inter- 
esting trends. It would appear that the job role classifications 
"principal investigator" and "project director" are concerned with the 
establishment and maintenance of facilitating events and conditions. 
k Scanning down the table of job role makes clear that responsibility 

is increasingly held for products with less and less emphasis on the 
l production events and conditions. This table suggests that there are 

differences between the job role categories utilized and, in fact, 
there may be differences in the functions performed by the individ- 
uals filling these job role categories. 

If individuals in various job roles are differentiated on the basis 
of establishing and maintaining facilitating events and conditions, the 
question might be raised, "Is it possible to differentiate between job 
roles on the basis of the level of outputs produced?" Table 12.6 



Table 12.6 The Level of Outputs as ordered by Job Role 



Job 

Role 


Output Level 


Fooal 


Component 


Facilitating 


Principal Investigator (I4) a 
Project Director (12) 

Top Level Project Manager (25) 
First Line Supervisor (5) 
Professional Staff Member (48) 
Clerical Support Staff (1) 
Technical Support Staff (4) 
Advisor or Consultant (6) 


6 (14%) b 

7 (15%) 
10 (18%) 

1 (10%) 
14 (15%) 

2 (22%) 


2 ( 5%) 34 (81%) 

1 ( 2%) 38 (bJ%) 

16 (29%) 29 (53%) 

3 (30%) 0 (60%) 

29 (31%) 50 (54%) 

1 (100%) 

1 (11%) 6 (67%) 

1 (20%) 4 (81%) 



a Tne number of individuals classified within the category. 

^The number 6 represents the exact number of Outputs classified as Focal within 
the Principal Investigator. For purposes of discussion these raw data have been 
transferred to percentages in order to account for the different numbers of 
individuals within each category. 
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organizes the output data as classified by the focal, component and 
facilitating levels by the same set '>f job role categories introduced 
previously. By scanning the column showing the percentage of focal 
outputs it can be seen that there is little deviation on the basis of 
Job role. With respect to the component output level, however, it is 
clear that personnel in the first two categories of job role have a 
very small percentage of their outputs classified as component. The 
next three categories of job role, however, would appear not to vary 
significantly in the percentage of outputs so classified. The last 
column of outputs shows clearly that the principal investigator and 
project director job roles have a large percentage of the^r outputs 
classified as facilitating. While top level project managers, first 
line supervisors and professional staff members also have more than 
50% of their outputs so classified, the hypothesis raided in diTcussing 
the data within Table 12.5 would still appear to be viable. It is 
interesting to note that the technical support staff category also has 
a very large percentage of outputs classified as facilitating. The 
percentages associated with advisor and consultants also begins to 
provide additional clarity in terms of the utilization made of the 
individuals , 

As was pointed out earlier within this discussion, the output 
data as ordered by output function within Chapter 9 indicated that 
more than 50% of the outputs identified across all 20 project sites 
were classified as, management . Table 12.7 organizes the output func- 
tion data as it distributes by the job role categories. Data from the 



Table 12.7 The Function of Outputs as ordered by Job Role 



Job 

Role 


Output Function 


Eolicy Setting 


Management 


Product! on 


Principal Investigator (14) a 


3 ( 7%) 


33 (79%) 


6 (14%) 


Project Director (12) 


3 ( 7%) 


35 (76%) 


8 (18%) 


Top Level Project Manager (25) 


- - 


31 (56%) 


24 (44%) 


First Line Supervisor (5) 


- - 


5 (50%) 


5 (50%) 


Professional Staff Member (48) 


- - 


34 (37%) 


59 (63%) 


Clerical Support Staff (1) 


- - 


- - 


1 (100%) 


Technical Support Staff (4) 


- - 


5 (56%) 


4 (44%) 


Advisor or Consultant (6) 




4 (81%) 


1 (21%) 



a The number of individuals classified within the category. 

^The number 3 represents the exact number of Outputs classified as Policy-Setting 
within the Principal Investigator. For purposes of discussion these raw data 
have been transferred to percentages in order to account for the different numbers 
of individuals within each category. 
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table indicate that policy setting occurs only in relation to the first 
two categories of job role. Also within those two categories the pri- 
mary emphasis is on management classified outputs. The top level pro- 
ject manager and first line supervisor appear to fairly well divide their 
production of outputs between those classified as management and those 
classified as production. 

Professional staff members eppear to have a greater percentage of 
outputs classified as production than any other job role category. The 
clerical and technical support staff data are self explanatory. Indivi- 
duals classified as advisors and/or consultants appear to deal primarily 
with management related outputs. As the table shows, all job role cate- 
gories have management classified outputs identified with them. It is 
alsj clear from the table that there is some degree of differentiation 
among the job role categories. For example, policy-setting as a class 
of outputs was linked to only the first two categories of job role. The 
table also shows that the percentage of management outputs tends to decrease 
from Principal Investigator to Professional Staff Member while the per- 
centage of production classed outputs tends to increase. 

It is at this point that these data begin to point to some of the 
ambiguities that exist in the data which make up job role descriptions. 

If similar types of outputs are of concern to all personnel irrespective 
of their job roles, to have classified such data on the basis of job 
role not only would have given a faulty picture of the actual nature 
of the data but, and perhaps more importantly, eliminated the possibility 
of looking at the similarities of the outputs across job roles. 

Table 12.8 organizes for the reader the Character of outputs as 
ordered by individuals’ job roles. Previous chapters have referred 
to the fact that there is an emphasis on technology classed outputs 
irrespective of project focus. This would also appear to be the case 
with the job role data. The only job role category which has less than 
50% of the outputs classed as technology is First Line Supervisor. 

Within Table 12.8 four categories of job role are listed which 
have none of their outputs classified as knowledge. The reader is en- 
couraged to review the remainder of the table from the point of view 
of these null sets and the sets which have very low percentages. It 
is of interest to conjecture about what the data would appear to be in 
terms of output character if this table could be reorganized utilizing 
such additional sorting categories as project focus ox setting. It is 
simply unclear as to whether or not any further differentiation of the 
data would be forthcoming with such a further split of the data. 

Thus far in the discussion of this special issue, output data has 
been reviewed as it is clustered by the various job role categories. 

The data presented som^ potentially useful differentiation between job 
roles when utilizing structure, level, and function outputs for compari- 
son. It also seems clear from the data that there is no sharp distinc- 
tion between individuals who might be called managers and individuals 
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Table 12. b The Character of Outputs as ordered by Job Role 



Job 

Role 




Output 


Character 




Knowledge 


Technology 


Implementation 


Information 


Principal Investigator (l4) a 


7 (17%) b 


26 (62%) 


5 (12%) 


4 (10%) 


Pioject Director (12) 


7 (15%) 


23 (50%) 


9 (20%) 


7 (15%) 


Top Level Project Manager (25) 


1 ( 2%) 


36 (66%) 


8 (15%) 


10 (18%) 


First Line Supervisor (5) 


- - 


4 (40%) 


2 (20%) 


4 (40%) 


Professional Staff Member (48) 


2 (22%) 


70 (75%) 


7 ( 8%) 


14 (15%) 


Clerical Support Staff (1) 


- - 


1 (100%) 


- - 


- - 


Technical Support Staff (4) 


- - 


5 (56%) 


3 (33%) 


1 (11%) 


Advisor or Consultant (6) 




4 (81%) 




1 (21%) 



a The number of individuals classified within the category. 

^The number 7 represents the exact number of Outputs classified as Knowledge within 
the Principal Investigator. For purposes of discussion these raw data have been 
transferred to percentages in order to account for the different numbers of indi- 
viduals within each category. 



who might be called staff. Distinctions tend to be more subtle in nature, 
more or less sliding across job roles in a manner dependent upon the or- 
ganizer chosen. A natural question to ask next concerns differentiation 
in terms of the work requirement data. "Do the different job role cate- 
gories, in fact, differentiate themselves in terms of the work require- 
ments?" Once again, because of the vast number of work requirements 
data categories derived during the Oregon Studies, the question will 
be approached utilizing summarized data. Specific categories of stand- 
ards, tasks, etc. will not be used. Averages will be used instead. 

Table 12.9 provides data which indicate the average number of outputs, 
standards, tasks, and enabling knowledges, skills and sensitivities 
identified by each individual within the related job role categories. 

The average number of outputs identified with each job role decreases 
as the table is scanned from top to bottom. The distinction between 
standards of the process and output variety presents an interesting 
analysis. As Table 12.9 shows, the Principal Investigator category 
reported a fairly equal number of process and output standards. The 
Project Director category, on the other hand, is more concerned with 
process related types of standards. The latter concern also holds for 
individuals classified within Top Level Project Manager job role cate- 
gory. A shift, however, takes place for the job role category First 
Line Supervisor, with the emphasis at this point being placed upon out- 
put standards. Reviewing the rest of the table it can be seen that this 
is the only job role category which has reversed the emphasis from the 
process standard side to the output standard side. The remainder of 
the data within this table can be reviewed by the reader in similar 
fashion . 



Table 12.9 The Average number of Outputs and Work Requirements identified 
with individuals clustered by Job Role 







The Average 3 Number of 


Work Requirements 




Job 

Role 


OutDUtS 


Process 

Standards 


Output 

Standards 


Tasks 


Knowledces 


Skills 


Sensitivitie 


Principal Investigator (14) 


3. 0 


6.0 


5.5 


33.1 


9.3 


6.9 


7.0 


Project Director (12) 


3.8 


9.6 


5.6 


44.3 


12.8 


11.2 


9.7 


Top Level Project Manager (25) 


2.2 


7.6 


3.4 


32.6 


7.8 


7.4 


6.1 


First Line Supervisor (5) 


2.0 


8.0 


11.0 


40.2 


8.6 


12.2 


8.4 


Professional Staff Member (48) 


1.9 


7.7 


1.3 


31.2 


8.4 


8.6 


5.6 


Clerical Support Staff (1) 


1.0 


4.0 


0.0 


29.0 


6.0 


6.0 


2.0 


Technical Support Staff (4) 


2.3 


5.5 


1.8 


33.5 


13.8 


19.0 


8.0 


Advisor or Consultant (6) 


0.8 


4.0 


0.8 


8.7 


3.0 


3.0 


1.2 


a The average figures were derived by dividing the raw number of outputs by the number of 
individuals within a job role category, as an example. 



In an attempt to further clarify the data presented within Table 
12.9, Table 12.10 restructures the data in terms of the job vole cate- 
gories to indicate the average number of work requirements indicated 
per output. Once again, in reviewing the process output standards col- 
umns, it is interesting to note that the individuals classified as 



Table 12.10 The Average number of Work Requirements identified for the 

average number of Outputs identified for Categories of Job Role 



Job 

Role 


The A} 


£era&e a Number of Work Requirements per OutDut 


Process 

Standards 


Output 

Standards 


Tasks 


Knowledges 


Skills 


Sensitivitie 


Principal Investigator (14) 
Project Director (12) 

Top Level Project Manager (25) 
First Line Supervisor (5) 
Professional Staff Member (48) 
Clerical Support Staff (1) 
Technical Support Staff (4) 
Advisor or Consultant (6) 


2-0 1.8 11.0 3.1 2.3 2.3 

2-5 1.5 11.7 3.4 2.9 2.6 

3.5 1.5 14.8 3.6 3.4 2.8 

4.0 5.5 20.1 4.3 6.1 4.2 

4.1 0.7 16.4 4.4 4.5 3.0 

4.0 0.0 29.0 6.0 6.0 2.0 

2.4 0.8 14.6 6.0 8.3 3.5 

5.0 1.0 10.9 3.8 3.8 1.5 



a The average figures were derived by dividing, as an example, the number of process standards 
identified in Table 12.9 by the number of Outputs identified within the same Table. 




Or 

rtr.. 






205 



F i r« t Line Supervisor are the only group, in terms of the work require- 
ments, which places heavy emphasis upon output standards. This may sug- 
gest that they were serving in the role of "quality control agent," and, 
in that role, having a high level of concern with processes. It is in- 
teresting to note also that individuals classified as Professional Staff 
Member indicated less than one output standard on the average for each 
of the outputs with which they were associated. 

With the exception of the very narrow technical areas represented 
by the clerical and technical support staff there appears to be little 
differentiation in terms of the number of enabling knowledges, skills 
and sensitivities identified for each of the outputs produced by the 
various job roles. The data suggest that each job role relies on about 
the same range of knowledges, and exercises equivalent ranges of exper- 
tise for any given output. The total number of outputs which can be 
associated with each role is not known at this time. With a more exten- 
sive set of data it may well be that differences occur at the level of 
numbers of outputs attended to by each job role as well as the specific 
categories of knowledges, etc. associated with job roles and outputs. 

An examination of categories of knowledges associated with a job role 
at this time unfortunately could do no more than point out the low 
frequencies within each cell entry. 

Data in each of the tables presented so far suggests that different 
clusterings of job roles can facilitate data analysis. From the point 
of view of planning a project effort it appears job roles are, in fact, 
not well defined and tend to slip and slide by Level, Function, Structure 
and Character of output. The data, then, would appear to have a signi- 
ficant bearing on the questions of management and organizational struct- 
ures. For example, if everything done by a "manager" is defined as 
management and the output of other "non-management" types as production, 
the resultant data would appear quite different from that produced by 
the Oregon Studies. To build an organizational structure around a con- 
cept such as job role then, might prove to be an inadequate procedure. 
This possibility was alluded to any number of times by the interviewees 
as they indicated that the charts that represented the line and staff 
relationships did not factually represent the sets of functional work- 
ing relationships that endured through time. It would appear from the 
Studies’ data that organizing by job role is subject to question as an 
effective procedure. On the other hand it would appear that job role 
might be a convenient organizer around which to discuss gross categories 
of activity as long as clarity exists, in rather precise terms, about 
what individuals who are categorized within a particular job role are 
to produce and how each of those elements influence or fit into the 
overall effort. It is possible that outputs might eventually come to 
serve the role of organizer now served by job role if the Oregon Studies 
data proves valid and reliable over time. 



Special Issue IV: 



The Utility of the Classes of Data 
and their Interlinkage 



The purpose of the present discussion is to give perspective to 
the classes of data available from the Oregon Studies in terms of their 
utilization by trainers and practitioners in the field of educational 
RDD&E. Toward that end discussion of the issue is organized around (a) 
the substantive utility of the classes of data available as independent 
sets and in combination, and (_b) the linking of '.he data sets to a pro- 
gram context. 



Substantive Utility of the Data Sets Available 

The major premise of the conceptual framework within which the 
Oregon Studies rests is, that people perform work within contexts to 
produce outputs or outcomes . Accordingly, the data produced by the 
Oregon Studies were collected and ordered into three broad classes or 
data sets, i.e., (a) context linked, (b) people linked, and (£) output 
linked. Figure 12.1 illustrates the nature of the interdependent rela - 
tionships of these setc . x As illustrated, the information contained 
within the data sets has specific meaning to the working contexts from 
which it was obtained. The present discussion focuses on viewing the 
data as information having relevance and meaning to trai :ers and pro- 
spective practitioners. 



Utility of the Ind e pendent Data Sets 

The independent data sets provided by the Oregon Studies have a 
utility relative to the development of programs which runs parallel to 
that of mapping the domain of educational RDD&E. It has been argued 
that the parameters of a domain can be described in terms of (a) con- 
text (the territory), (_b) people (the inhabitants), and (O outputs 
(the artifacts produced). Given the knowledge base contained within 
each of the independent data sets of the Oregon Studies, the user may 
judge the adequacy of his resources and the degree to which he can 
attend to any or all of the variables identified within the constraints 
of an academic degree program or work context time line. 



iFigure 12.1 illustrates the functional or dependency relationships of 
the sets of data illustrated in Figure A. 2 of Chapter 4. An exLended 
discussion of the data sets also found in that chapter. 
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Fig 12.1. The Nature of the Interdependent Relationships of the Data Sets 



In a substantive sense, context-linked variables provide the user 
with information regarding the focus, setting, size, etc., of educational 
RDD&E. Such information can aid the user in planning a program which, 
for example, prepares a trainee or novice to (_a) work in lab centers, 
colleges/universities, or in public schools, (b_) to work as relatively 
independent agent or as Integrated team member in a large organizational 
structure or project, (c) to work as a researcher, developer, etc. Each 
of these variables has potential utility for identifying such things as 
inservice training sites or the breadth and depth of tool skills which 
must be possessed by the trainee or novice. People-linked data explicate 
for the user knowledge of such variables as relevant backgrounds and 
experiences, job roles being performed, and degrees held, etc. Here 
the uset is guided in his planning by those elements that functioning 
practitioners feel are important. In addition, the descriptions of 
those functioning practitioners may suggest to the user the need to 
plan or design programs which systematically introduce the trainee or 
novice to persons representative of those with whom he is ultimately 
apt to be working. Output-linked data, on the other hand, introduces 
the trainer or practitioner to the broad range of things considered 
to be relevant and important outcomes of focused work effort. The vari- 
ability of these outcomes, or outputs, along with the standards and work 
requirements associated with their generation should surely indicate the 
range of resources which might be brought to bear on the program, pro- 
duction, management and organizational tasks. 

In sum, the independent data sets permit the user to view system- 
atically the range of variables around which training or practice might 
be focused. It is possbile, also, to identify those variables having 
obvious implications for program format decisions (e.g., classroom vs. 
inservice training) . 



Utility of the Data Sets in Combination 

Independent consideration of each of the data sets provides one 
level of guidance to planning and conducting program activities. Given 
the array of data in each independent case, however, ♦ he prospective 
user is left to decide for himself what is important within each class 
of data, and to select those elements considered relevant to a speci- 
fic program need. Figure 12.1 implies that the user may be systematically 
assisted in giving focus to his selection of elements. Such assistance 
is provided by combining the context, people, and output-linked data 
sets in various ways. 
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One way in which the available data may be viewed in support of a 
program or project design can be diagrammed as follows: 




Since the nature and scope of job roles, for example, may vary according 
to the contexts in which they are carried out, the diagram suggests that 
the job roles of people be associated with the various contexts in which 
they occur. Thus, for a user choosing a particular focus and a particu- 
lar setting in which his program may be expected to work, the range and 
scope of people-linked data may be more discretely defined. For example, 
a "manager" in a large public school will quite likely perform a dif- 
ferent set of work activities than will a "manager" in a lab center. 

By the same token, a "producer" may find his role narrowly defined in 
a large public school context, while in a small college setting he may 
find that his role is broadly defined. 

The general utility of linking Context and People data for planning 
purposes lies in the users ability to more specifically identify job 
roles within a particular context or set of contexts. Just as important 
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as having a particular ability is knowing where and how the exercise 
of that ability may be relevant or dysfunctional. In some contexts 
it is possible that the exercise of a particular ability may be inap- 
propriate until other abilities are first exercised. In any event* 
the user may specify one or more contexts which have relevance for his 
efforts and about which information is needed and obtain three general 
sets of data, (a) the job roles found in such contexts, (_b) the work 
activities associated with those roles, and (c_) the backgrounds and 
characteristics of people who carry out those roles. (See Figure 12.1) 

Another way in which the user may view his program relative to 
the data available from the Oregon Studies is diagrammed as follows: 



PROJECT 
CONTEXT DATA 



Focus 

(RDO/E) 




Here again, the user has the option of specifying a focus, as an example, 
for which he intends to train personnel. The outputs associated with 
those contexts may then be identified along with the work requirements 
linked to them. The advantage here is twofold. First, the range of 
outputs has been reduced by limiting the range of contexts. Second, 
the range of work requirements associated with a particular output has 
been reduced to only those having significance for producing that out- 
put within the selected context. Thus the user has available a more 
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precise definition of the output, and the standards, enablers, and 
tasks associated with its generation, by virtue of the fact that it 
derives from a particular context* 

Yet another way of viewing the data available from the Oregon 
Studies is in relation to people and the outputs they produce: 




Here, the user may, for example, specify job roles being trained for 
and obtain three sets of data having relevance for training, (a) the 
general w ^rk requirements associated with the job roles, (b) the outputs 
and their characterise 3 associated with people in those job roles and 
(£) the work requirements for producing those outputs. Of special signi- 
ficance here is the ability to discriminate the nature of the outputs 
for which people in varying production and management roles have respon- 
sibility. Thus, the trainer is able to focus his training of M raanagers, M 
for example, on the generation of those outputs for which a manager has 
unique responsibility in projects. 

For the user who wishes to design a program or project that takes 
into account the interdependent nature of the realities encountered in 
world of educational RDD&E practice, all three major classes of data 
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generated by the Oregon Studies may be considered in combination as in 
Figure 12.1. Simply stated, the individual following this model can 
specify the parameters of contexts for which he chooses to prepare 
people or in which he is functioning. He can choose from within those 
contexts the related roles which are to be provided for over time, 
identify the work requirements related to those roles, identify the 
outputs produced bv such roles, and determine the enablers, tasks, and 
standards associated with those outputs. Implicit in this structure 
is the assumption that the ultimate success of a program is measured 
in terms of whether practitioners can fulfill the requirements of the 
roles they assume in various contexts by producing the outputs required 
of them to the standards of excellence demanded. The practitioner will 
find these data and their interdependencies useful in terms of predict- 
ing what may occur within a set of known factors as well as in identify- 
ing appropriate staff for his specific activities and intended outputs. 

The 3 data sets of the Oregon Studies have utility not only inde- 
pendently but as interlinked sets. While the data do not encompass the 
entire field uf educational RDD&E, they do provide an initial basis for 
making decisions. While the specific data lists are still limited, the 
structure which surrounds such data is already operational. It is hoped 
that this discussion will help the reader to determine whether the ques- 
tions which he has undoubtedly generated can be dealt with by the Oregon 
Studies system of data management. 

Table 12.11 reviews for the reader each of the data sets employed 
by the Oregon Studies, and the source, form, and extensiveness of that 
data. In addition, the table indicates where in the series of volumes 
reporting the Oregon Studies discussion of the data relating to each 
set may be found. 
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TABU n. 11 



A Description ol the Data Srta Employed In the Oregon Studies am) Where They Can Be Found 



Clnue of Data 


Source 


Fora 


Eaten* lveneaa 


Locat Ion 


Context Data 


Focuu 


On-alte vlaitatlon 


Criteria claaall lcat Ion 


4 Claaalf lcat Iona 


Volume 1 and IV 


Setting 


On-alte vlaitatlon 


Logical claaalf lcat ion and 


4 Closalf lcat lone of 


Ch, 12 V. 1 and V. XV 






written deacriptlon 


20 projects 




Size 


On-a lte visitation 


Numerical coapoalte of 


7 Classifications of 


Ch. 12 V. 1 and V. IV 




and document a 


funding level, duration 


20 projecta 




Content 


Documents 


Written deacrlpt 4 on 


20 projecta 


Voluae IV 


Dynamics 


On-alte vlaitatlon 


Wilt ten deacriptlon 


20 projecta 


Voluae IV 


Reaourcea 


On-alte vlaitatlon 


<>ieatlonnalra 


8 aajor catagorlea 


Chapter 6, Vol * 1* 



People Data (134 Intrrvlavcea) 



Job Role 


On-alte vlaitatlon 


(^leatlonnalre ranking and 
written description 


6 product lon/aanage- 
aent roles 


Chapter 6, Vol. 1* and 
Voluae IV 


Work Actlvltlea 


C.i-eite vlaitatlon 


Quest tonne lea responses 


8 aajor clusters, 
70 lteaa 


Chapter 6, Vol* I* and 
Voluae IV 


Degrees, Exper- 
iences, Charac- 
teristics, etc. 


On-alte vlaitatlon 


Questionnaire reaponaea 


23 variables 


Chapter 6, Vol* 1* and 
Voluae IV 



Output Data 



Identified Outputs 


On-alte visitation 


Llata «nd aapa as verified 
by each project 


962 Outputs 


Chapter 7, Vol* I and 
Volume IV* 


Primary Categories 


Identified outputs 


Llata of discretely dlf- 
faraat outputs 


326 Categories 


Ch* 7 and Appendix to 
Voluae I* 


Cluster Categories 


Identified outputs 


Llata generated free cri- 
teria claaalf lcat Ion 


51 Categories 


Ch. 7 and Appendix to 
Voluae I* 


output Structure 


On-alte visitation 


Criteria classification 


3 Claaalf lcatlona 


Ch, 9 and Appendix to 
Vol. I and Vol. IV* 


Function 


.1 II 


n n 


ii ii 


ii ii •• 


Level 


II II 


ii »» 


n ii 


H ii ii 

. n 


Character 


II II 


« i» 


4 Claaalf lcatlona 


ii tt n 


Standards 










Stateaenta 


On-alte visitation 


Recapped llata froa tapes 


1146 stateaenta 


V, IV Profile Appendices 


Categories 


Interview stateaenta 


Llata generated froa cri- 
teria classification 


2 structures and 
79 categories 


Ch. 6 and Appendix to 
Vol. 1 and Vol* IV* 


Taaka 










Stateaenta 


On-alte visitation 


lecapped llata froa tapes 


3722 atateaenta 


V* IV Profile Appendices 


Categories 


Iitervlev statements 


Llata generated froa cri- 
teria claaalf lcatlon 


20 clusters and 280 
categories 


Vol. I and Vol. IV* 


Cnablera 










Stateaenta 


On-alte visitation 


Recapped llata froa tapes 


2497 statements 


V. IV Profile Appendices 


Categories 


Interview stateaenta 


Llata generated froa cri- 
teria claaalf lcatlon 


3 structures and 
136 categories 


Ch. 0 and Appendix to 
Vol. I and Vol. IV* 



* Aaterlaka lndlcata that additional data exist la the (lira for which exhaustive analyaea ware not aada . Tha data 
flies thua represent a major output of the Oregoa Studies. 
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Chapter 13 



IMPLICATIONS FOR CONCEPTUAL AND METHODOLOGICAL DEVELOPMENT 



The preceding chapters of this volume have led the reader into and 
through the empirical maps of the domain of educational RDD&E as plotted 
by the Oregon Studies. From the opening challenge of Chapter 1 to the 
closing issue of Chapter 12, the emphasis has been on discussion of the 
constructs involved, the variables associated with them, and the data 
collected about them. In addition the reader has been repeatedly cautioned 
about the limits of the data and the degree to which varLous aspects of 
the data can be generalized. The purpose of the present chapter is not, 
therefore, to address strengths and weaknesses but rather to make explicit 
what appears to be needed, conceptually and methodologically, to bring 
further clarity to the nature of educational RDD&E and the training which 
supports it. The reader is forewarned, however, that understanding the 
following content will require basic familiarity with the content of 
Chapters 1-4 of this volume. 



Problem Areas for Further Work 



In addrrising the need for further conceptual and methodological develop- 
ment it must be remembered that there was interaction between the method- 
ological framework of data generation in the Oregon Studies and the con- 
ceptual framework which guided it. Thus it seems appropriate to discuss 
these developmental needs in terms of what kinds of information and under- 
standing (with respect to educational RDD&E) are needed beyond what the two 
frameworks now offer. The needs seem to reside in, but are certainly not 
limited to, five problem areas; (a) the explication of "operations," (b) the 
resolution of "level" problems, (c_) the refinement of the linkages of data, 

(cp the extension of the frameworks to integrate additional phenomena ob- 
served in the field, and (e) the refinement of the data generation strategy 
to increase its efficiency. Each of these is discussed separately. 



"Operations" as a Problem Area 

The basic elements which conceptually define project operations, l.e., 
actions, tasks, and activities, could not be discriminated within the time 
frame of the Studies. As a construct, operations data were to describe what 
it takes to produce various outputs. The elements of operations data were 
designed to explicate patterns associated with those efforts. This frame- 
work seems to remain sound and indeed could make a significant contribution 
to understanding operations at a level other than that of the nature of the 
work. Methodologically it is possible that inability to discriminate among 
the elements was due to potentially "circular" (or linear) conceptual defi- 
nitions, i.e., an "action" is a unit of a "task" which is a unit of "activity," 
etc., which do not provide a stable reference point. 
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Conceptually, the first task would seem to be one of examining the output 
and work requirements data gathered thus far for any new insights which 
mieht emerge. From that point, two alternatives are seen: (a) redefini- 

tion of the conceptualized elements of operations such that they retain 
the dependency linkages but have discrete discriminators included, or 
(b^) reconceptualizing the taxonomy used to describe operations. 



"Level" as a Problem Area 

The nature of the problem just described for operations is one 
example of a "level” problem, but it was discussed separately because of 
the implications for construct redefinition which seem to be involved. 

Other level problems, however, have a different kind of importance. The 
examination of outputs, for example, suffered a similar "level" problem. 
Conceptually, four levels of output orientation had been specified, i.e., 
facilitating, component, terminal, and focal. Operationally the terminal 
classification had to be abandoned in the absence of a sufficient base on 
which to establish stable decision rules. The loss of this capability 
hampered the capacity of the methodology to display the interdependencies 
of outputs at the sought-after level of detail. As a result, the rigorous 
testing of the nested set hypothesis regarding educational R,D,D, and E was 
somewhat impaired. The importance of this is seen in the potentiality 
that such clarification holds for a disciplined examination of organizational 
structures when viewed from the vantage point of their outputs. The examina- 
tion of output maps which order outputs on the basis 01 their interdependent 
relationships within a project may aid in the conceptualization of a taxon- 
omy which better describes output level. From a methodological point of 
view, the rules for level classification would have to be revised accordingly 
and the relationship between the classification of an output and the develop- 
ment of an output map clearly delineated. 

Another "level" factor relates to initial output identification. From 
a methodological point of view it may be possible to identify "an output" 
for nearly any task mentioned by an interviewee. Thus far in methodologi- 
cal development, however, the guidelines recommend identifying such outputs 
only to the extent that an informant perceives a task or set of tasks as 
having a discrete output. The full implication of this strategy as it 
relates to mapping the domain needs exhaustive study. Concretely, as a 
result of the strategy followed the data of the Oregon Studies may appeal to 
vary from oth^r studies in terms of inclusions and comprehensiveness. 

Cursory analysis of this phenomenon indicates that it is likely a function 
of differences in where a particular data element appears, for example in 
task data rather than output data or vice versa. The point is, there would 
appear to be added power in conceptualizing, and handling methodologically, 
the identification of the outputs of any task even though the task ^.cself 
was perceived as relating to a higher order output. By discriminating be- 
tween perceived outputs and task outputs an output map could be constructed 
which could give added dimension to the operational structure of a project. 



The nested set hypothesis holds that some research, development, diffusion, 
and evaluation strategies are employed within each of R,D,D, and E, A com- 
plete discussion is contained in the Schalock-Sell conceptual paper in 
Volume III of the Oregon Studies. 






Data Linkages as a Problem Area 



There would appear to be two major considerations with respect to 
the linkages of data in the Oregon Studies. Specifically these involve 
(a) linkages of the Function variable and (b) linkages of enabler and task 
data. The Function classification has, methodologically, been applied to 
outputs only. While this is not necessarily at variance with the concep- 
tual framework, it provides only limited understanding of Function as a 
variable within project operations. Presumably, other variables could be 
linked to Function as well, and a conceptual effort to identify those 
applications is needed. Given such applications, careful consideration 
would have to be given to the method by which such classifications could 
be reliably and validly determined. 

The second set of linkages which seems desirable is between enablers 
and tasks. The ability to link enablers to tasks would provide the trainer 
with a step-by-step level of detail in producing outputs that may add to 
the utility of the data. It is possible, of course, to elicit that detail 
by interview, but the redundancy which occurs from one task to another, 
given the same output, may yield ‘‘convenient’ 1 response data rather than 
discrete data. Indeed, experience showed that interviewees tended to 
report previously given enablers for each task. Such redundancy became 
obtrusive with the result that respondents acknowledged they were beginning 
to give “convenient” responses. Conceptually, there needs to be a review 
of the gains and losses in establishing this linkage in data collection 
as opposed to creating it in data reduction. Methodologically, added 
testing of various alternatives may produce the linkage at a data collection 
level in such a way as to eliminate the obtrusiveness cf redundancy in the 
eyes of the respondent. 



Extension of the Framework as a Problem Area 



Two immediate extensions of the framework are apparent from the data 
collection efforts and the data collected to date. These include a) person- 
ality characteristics relative to / people and b) the acknowledgement of the 
changing character of an output when viewed according to its use. 

Attitudes and personality characteristics viewed as essential enablers 
appeared in the sensitivity data collected by the Oregon Studies. From the 
conceptual point of view, consideration needs to be given as to the legiti- 
macy of personality characteristics as an enabling class cf data or as part 
of the people-specific data. On the one hand, that class of data has power 
when connected to outputs in describing those characteristics essential to 
achieving an output. On the other hand, as part of people-linked data it 
provides a means for more exhaustively describing the people within a con- 
text. From a methodological point of view, if personality characteristics 
are retained in the enablers dat/. a new category set of data needs to be 
established, and the interview strategy needs to be expanded to focus on 
that category the same level as was the case with sensitivities. Should 
such data be conceptually linked to people, another means of systematically 
collecting the data needs to be developed. 
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With regard to outputs, the nuances and technicalities involved with 
classifying the Character of a thing as an output of a project is diffi- 
cult to manage cognitively in the absence of a construct which distinguishes 
the output per se from the use to be made of it (as input). For example, 
the decision rule is to regard a project's production of new knowledge as 
an instance of the output of research and, in addition, to label that 
output as having the Character of Knowledge. On the other hand, its use 
within that project (or by others) may cause it to be regarded as Infor- 
mation (which in output terms is an output of Evaluation). Considerable 
clarity could be brought to the interacting nature of RDD&E as a nested-set 
if the transiency of the classification given the output of each is con- 
trolled by a dual classification system. The second classification would 
deal with the use or intended use of the outputs. A conceptual argument 
for this point of view needs to be developed and the concept integrated 
into the framework in such a way as to show how clarity is achieved by 
such process. Methodologically, the same character classification labels 
would be used for the dual classification, but discriminators, strategies, 
and decision rules for identifying the uses of outputs (giving them "input" 
identity) would need to be established. 



Data Generation Strategy as Problem Area 

Reference has been made to the transportability of the methodology as 
a feasible, cost effective means of further data generation. Involved 
here are issues of economy relative to training persons to collect data by 
way of the methodology, the efficiency with which data can be collected 
and processed, and the trustworthiness of the information load contained 
in that data. 

In present form, the methodology is reasonably operable and straight- 
forward relative to the data collection process. Whether it be in interview 
or questionnaire form,' it is anticipated that others may be efficiently 
trained to collect the data in accordance with the methodology. There is 
reason to believe, however, chat the data reduction process remains complex 
and difficult, with the level of detail given in the guidelines not yet 
sufficiently explicated to change this condition appreciably. 

Judging the efficiency of the data collection strategy as such must be 
referenced against short term and long term data needs. The interview 
;itrateiy provides a means of controlling against idealized or "obvious 
response" data such as becomes a problem in interpreting questionnaire 
data. As long as the premium is on describing or mapping the realities 
which inhere in projects, there are acceptable levels of benefits to be 
derived from use of the more time consuming and costly interview strategy. 
For the short term then, including at least one more data collection effort 
to test methodological revisions and the stability and exhaustiveness of 
current data elements, the interview strategy is the more productive. 

Long term data needs which focus on the probable existence of and 
degree of emphasis given various identified characteristics of the domain 
of educational RDD&E over time, however, demand a less costly methodology. 
Toward that end careful consideration needs to be given to the design of 
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a survey instrument (or instruments) based on the units of analysis and 
elements of data prescribed by the conceptual framework and which main- 
tains the integrity of the linkages between data sets. To the extent 
possible, the measurement scales used should reflect the emphasis placed 
on a phenomena as opposed to the single occurrence of the phenomena , and 
the concept of open-endedness of the categories should be maintained. In 
addition, the procedures and decision rules which identify and specify 
the response range expected from each individual within a surveyed context 
need to be established, 



Conceptual and Methodological Adequacy in Perspective 



The intended interactions of the conceptual and methodological thrusts 
of the Oregon Studies have been described and referred to throughout the 
present volume. In retrospect, these intended interactions have provided 
in large measure (and as expected), the real strength of the Studies. The 
conceptual framework provided a stable benchmark for the initial relating 
of descriptive elements of educational RDD&E to each other. As the con- 
cepts were verified or were proven to be idiosyncratic, at least in part 
because of the inaccessability of data, the methodology gave further shape 
to the framework. As a result, a conceptual framework has evolved which 
shows significant promise as a functional basis on which to give shape and 
direction to an emerging discipline. Concomitantly, a methodology has 
evolved which has been tested conceptually and empirically as a means of 

identifying parameters of the domain to be served by such a discipline. 

To the extent that existing practices in the projects described to date 
reflect reasonably the practices likely to continue for the present and 
near furture, the data generated by the application of the methodology 
represents the most comprehensive and potentially useful description of 
educational RDD&E activities available today. Further development and 
interpretive reporting can support diffusion of the results of the Oregon 
Studies to the field, but in the final analysis their utility will be 
measured in terms of the responsive use made by the field of the con- 
structs, of the information base contained within them, and of the de- 
gree to which the extension of data in depth is demanded. It is hoped 
that the contents of this chapter build on the integrity of efforts to 
collect data and lay a relevant and important foundation for perceiving 
critical tasks ahead. 
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Chapter 14 



IMPLICATIONS FOR TRAINING: HIGHLIGHTS OF THE DATA 



It will be recalled that while the Oregon Studies were designed to 
serve a basic science function they were initiated to serve the needs of 
training. As such they were among a number of research and development 
efforts that in combination were designed to reshape the nature of train- 
ing programs in educational RDD&E. The present chapter is the first of 
two chapters in the present volume that attend directly to the implica- 
tions of the Studies for the design of training programs. 1 

Since a primary aim of the effort to reshape training in educational 
RDD&E was to create a "data dependent ’ model of training, and since the 
Oregon Studies were designed to contribute to the data base needed to 
operate such a model, the two implications chapters that have been pre- 
pared are data oriented. The present chapter represents an attempt to 
pull together and highlight data, from this point of view, that appear 
in various forms and places throughout the report of the Studies. Two 
different kinds of data have been highlighted: (a_) the data reported in 

Chapters 6 through 12 of the present volume; and (b) the data reported 
in each case profile in the form of "implications for training" that 
emerged from informal discussions and observations while in the field. 

The first of these sets has been organized around a series of questions 
that might be asked by a person who is contemplating the Held of educa- 
tional RDD&E as a place to work. The second has been summarized in 
simple narrative form. The chapter closes with a discussion of the way 
in which the various Volumes in the series of Volumes reporting the 
Oregon Studies might be used in support of training efforts. 



Cross-Project Analyses 



In order to highlight data from the Oregon Studies a perspective 
was chosen that would allow for a relatively uncomplicated, straight- 
forward discussion of what the data might mean to a select reader. In 
service of this aim the following paragraphs are directed toward answer- 
ing a set of questions generated to reflect the concerns of a novice 
entering the field of RDD&E. The same approach could be used to serve 
other points of view such as that of the trainer, the practitioner, or 



1 Obviously, the full range of implications of the Studies for training 
cannot be dealt with in the space of two chapters. Nor can they be dealt 
with in service to all potential consumer groups. The limitations of the 
report of the Oregon Studies in this regard, and what can be done to 
overcome these limitations, are dealt with more fully in Chapter 16. 



Question 1. What kinds of people will I work with in educational RDD&E? 

(Ch 6) 

One of the striking characteristics of educational RDD&E personnel 
is the fact that most of them are male. Two out of every three people 
working in the general field are men. The lone exception to this general 
trend appears to be in diffusion. In that area, at least for the five 
diffusion projects studied, the majority of personnel are women. Dif- 
fusion personnel also tend to be older. Those who seek to work with 
younger staffs should look to the area of research. Those who value 
experience most highly will find personnel in the development projects 
to be older and more widely experienced than in any of the other areas. 



Question 2 . What kind of credentials do I need to work in the field? 

(Ch 6) 

All of the people who work in educational RDD&jrf, according to the 
Oregon Studies sample, hold at least a bachelors degree. Far greater 
than half hold either a masters or doctorate. In research projects and 
evaluation projects almost half the people surveyed held doctorates. 
While people in the field indicated that the work they did often did 
not require the possession of these degrees, they also indicated that 
the political realities were such that they were a needed credential. 
Most of the major job roles in the field, such as project investigator 
or project director, were held by those with the highest degrees. These 
same people were generally the ones who published articles and papers. 
Not everyone in educational RDD&E publishes. Over half the people sur- 
veyed had no publications. On the other hand, however, some people had 
more than 16 publications. 



Question 3 . How much money will I make? (Ch 6) 

Some people in development and diffusion projects earn more than 
$29,000 a year. Graduate students working as research assistants and 
part-time workers make less than $5000 a year. In general, most of the 
people earn between $12,000 and $17,000 a year. The amount of money 
earned depends on a number of factors; primarily experience, training, 
and academic degrees. 



Question 4 . Do I have to belong to any particular professional organi- 
zations? (Ch 6) 

No. Most of the personnel in the projects surveyed belonged to no 
professional associations. Those who did belong were members of AERA 
or NEA primarily, although a number of other associations were mentioned. 



Question 5 . What should I get a degree in? (Ch 6) 

Since at least a Bachelors degree is in apparent prerequisite to 
employment, it seems logical to assume that some kind of major or minor 
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area of study would also be important. In the real world, however, this 
does not appear to be the case. Members of RDD&E staffs majored in a 
wide variety of subjects. The largest proportion claimed majors in or 
related to Education or Psychology, but almost any area of specializa- 
tion seems to be acceptable. The important thing seems to be the degree. 



Question 6 . What kind of experience do I need? (Ch 6) 

Many people working in educational RDD&E have had no previous exper- 
ience. Experience in teaching at various levels and in working in educa- 
tion generally is fairly common, but even those working in the field 
don f t consider experience in RDD&E essential to much of the work done. 



Question 7 . What kinds of resources will I need or be using? (Ch 6) 

The most commonly used support service in the field of educational 
RDD&E generally is typing. Computer use, library, audio visual equip- 
ment and the like are other services provided by support staff. Person- 
nel in projects use such things as desk calculators, dictating equipment, 
keypunch machines and other similar kinds of equipment. 



Question 8 . What will I get paid for? (Ch 7) 

People working in educational RDD&E are responsible for producing 
a variety of products, causing certain events to take place, and creat- 
ing conditions that help to make things happen. All of these things 
combined represent the outputs of work within the context of education- 
al RDD&E. Most of the work is devoted to the production of products. 
The most common products produced are written reports of various kinds, 
project schedules and timelines, data about a variety of things, and a 
wide range of instruments and tools. The kind of products produced 
depends upon the focus of the project producing it (research, develop- 
ment, etc.) and upon the purpose of the particular project within that 
focus. When personnel are not engaged in producing products they are 
usually engaged in the execution of events, for example, recruiting 
staff, collecting and analyzing data, or adapting materials, or engaged 
in the creation of conditions. Examples of conditions that frequently 
have to be created in the world of RDD&E are good work atmospheres and 
cooperative relationships. 

The outputs of work within educational RDD&E generally play dif- 
ferent roles in different projects. Sometimes a given product, event, 
or condition is the main, or focal, output that a project is attempting 
to produce, sometimes it is needed as a component of the focal output, 
and sometimes it is used only to help or facilitate the production of 
other outputs. Some outputs also are required for the purpose of set- 
ting policy for a project, others for managing the project, and still 
others for carrying out production. The important thing to realize is 
that all work in educational RDD&E is directed toward the production of 
specified outputs that have specified standards, and that successful 
work results in the production of outputs that meet those standards. 
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Q uestion 9 . 



What kind of work will I have to do? (Ch 8) 



For any given output, a staff member or group of staff members have 
lu Lake advantage of the knowledges, skills, and sensitivities they pos- 
sess to perform the tasks that are required to produce the output to the 
standards set for it. Since a number of tasks have to be performed in 
the production of most outputs, personnel have to possess or acquire a 
wide variety of enablers (knowledges, skills, and sensitivities) in order 
to perform their work satisfactorily. In the performance of work it is 
important to realize that standards are set for how work is done as well 
as for the outputs of work. Some of the specific kinds of work educa- 
tional RDD&E personnel perform includes developing measures, creating 
cooperative relationships between agencies and people, working with 
schools, parents, and communities, thoroughly testing new ideas or in- 
novations and, almost always, writing reports about the work that was 
done . 



Question 10 . What do I need to know or be able to do to carry out edu- 
cational RDD&E tasks? (Ch 8) 

What a staff member has to know depends primarily upon what outputs 
he has to produce. There are some knowledges that appear to be needed 
more in some kinds of projects than in others. Almost all projects pro- 
duce outputs that require a knowledge about what a project is attempting 
to do, how the project is going to do it, and for or with whom it is to 
be done. Most also produce outputs that require a knowledge of educa- 
tional RDD&E related subjects and technical or discipline related 
subjects. 

The most commonly needed skill in educational RDD&E is the ability 
to write clearly, consisely, and understandably. Outputs produced in 
all project areas call for this skill to one degree or another. Other 
skills needed by educational RDD&E personnel involve helping people 
work together, making sense out of the various elements that constitute 
problems, planning, and using self discipline. In addition, research 
personnel rely heavily upon data handling skills, diffusion personnel 
rely heavily upon the ability to explain and justify what they’re doing, 
and evaluation personnel have to be good at analytical problem solving. 
Many other skills are also needed but those mentioned are the ones most 
frequently called upon. 

People working in educational RDD&E projects have to be sensitive 
to their own and others’ capabilities and limitations, and they need 
to be aware of and sensitive to existing value systems and the needs 
they and others have. They also need to be sensitive to the way the 
target audience of their work feels about the work, and about things 
in general. These sensitivities are the ones most commonly mentioned 
by people in the field. Many other sensitivities are needed as well, 
however. For example, in evaluation projects personnel need to be 
aware of the limitations of data and of the analyses that can be made 
with data. 
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Question 11 . What kinds of tasks will I need to perform? 



(Ch 8) 



There are many tasks that are performed in the field of education- 
al RDD&E . The kinds of tasks a given staff member will have to perform 
depends on the output he is trying to produce, but there are some tasks 
that are required often enough that nearly everyone in the field will 
have to do them at one time or another. One such task is the prepara- 
tion and assembling of the component pieces and parts of the contracted- 
for outputs, for example, fitting together various papers into a final 
report. Other tasks include making modifications that are needed in 
outputs, reviewing materials, negotiating contracts, getting people or 
agencies to cooperate in producing outputs, constructing and administer- 
ing tests, questionnaires and other data instruments, and communicating 
with colleagues or others to explain or critique or specify what has 
been or needs to be done. 



Question 12 . On what basis will my work be judged? (Ch 8) 

Standards of quality are applied to both outputs and the processes 
involved in producing them. For any output that is to be produced there 
are specific standards that have to be met before the output is con- 
sidered acceptable. The standards most commonly applied to outputs in 
the field of educational RDD&E demand that the goal established for an 
output be met, that other people in the project accept the output, that 
it is complete, that it gets accepted by whoever is supposed to use it, 
and that it is useful or valuable. Depending on what kind of project 
is involved, other standards for outputs frequently come into play. 
Personal satisfaction with an output is a frequently mentioned standard 
in research, as is a concern that the output functions as planned. 
Evaluation projects often use standards associated with being sure that 
outputs were designed correctly and were produced without error. 

In terms of the processes involved in producing outputs, common 
standards relate to whether the effort expended resulted in a favorable 
impact, whether people outside the project appeared enthusiastic about 
the work being done, whether deadlines were met, whether the project 
budget was lived within, and whether personnel met the criteria used 
for employment. Other process s standards mentioned frequently in various 
projects included the researcher’s concern with cooperation among per- 
sonnel, producing an acceptable level of output, and getting things 
done that are supposed to get done. Personnel in development projects, 
along with the standards mentioned, frequently show concern that no 
deficiencies show up in working on an output. Personnel in diffusion 
projects expect that people can see what needs to be done and do it. 



Question 13 . What will my job be? (Ch 6 and Profiles) 

The beginning staff member, with no experience and a bachelors 
degree, will most likely be considered simply a member of the profes- 
sional staff and hold the title of Instructor, Research Assistant or 
something similar, depending upon the agency in which he works. There 



art* a number of facLorn bucIi as age, experience, degree, pub 1 i cut I un.s , 
"connections" and the iike, not to mention demonstrated abiLlty, that 
determine the role of a particular staff member. Interestingly, when 
asked about job roles, as many regular project staff members identified 
themselves as holding a management level role as they did a production 
level role. There are apparently as many "bosses" in the field as 
there are workers. This unique role relationship is not as simple as 
it may appear, however, for the actual work done by managers is not 
greatly different from that done by other staff members. Both groups 
overlap in the performance of management and production work efforts. 



Question 14 . What do I need to know about the outputs of work effort? 

(Ch 9) 

Any given output assumes its own particular identity within the 
context in which it is produced. A guide or manual might be the focal 
output of, say, a development project and be only a facilitating output 
in an evaluation project. In one case it might serve the function of 
setting policy, while in another it might serve a management function. 

It simply depends. As a general guide it can be said that most outputs 
serve a management function and are used to facilitate other outputs. 
Very few products, even fewer events and yet fewer conditions are the 
focal outputs of any given project. Products and events serve mainly 
management and production functions, while conditions are mostly re- 
lated to management functions. 

Each output produced has its own special character. Some outputs 
represent knowledge, some are new or revised technologies, some provide 
information, and some are instances of implementation. The vast major- 
ity of all outputs produced in educational RDD&E, however, regardless 
of the focus of the projects involved, assume the character of a tech- 
nology. As might be expected research projects produce a high percent- 
age of knowledge outputs, development projects produce a high percentage 
of technological outputs, diffusion produces a high percentage of im- 
plementation outputs, and evaluation produces a high percentage of 
information outputs. Each project focus, however, appears to produce 
a range of outputs inclusive of most of these characteristics. 

Again the importance of an output depends entirely upon the con- 
text in which it is produced. The function an output serves, the level 
it assumes (focal, component, facilitating) and even its character are 
influenced by the setting within which it is produced and the purposes 
that a project is attempting to serve. 



Question 15 . How can I be trained to work within the context of educa- 
tional RDD&E? (Ch 15) 

At this time most universities offer courses specifically related 
to educational RDD&E topics. The most important training seems to be, 
according to those in the field, on-the-job experience. From the data 
gathered by the Oregon Studies, and the thinking of people who are 
involved in the field, new ideas are emerging with respect to training. 
These are summarized in Chapter 15, and, to some extent, in the section 
that follows. 



Highlights of the Data that Emerged 
from Case Profile Analyses 



The last chapter of each profile discusses training implications 
that emerged from the study of each project. These implications were 
obtained at the site from questionnaires and interviews, and from the 
overall impressions gained by the visiting data collection team. For 
purposes of the present chapter these data have been summarized in 
• narrative form. 

As might be expected virtually all the chapters on implications 
for training (18 out of 20) mentioned the need for specific training 
in research design and methodology, statistics, project management, 
systems, computer usage* and the like. In six of the 20 sites studied 
technical subjects were assigned the highest priority for training. 
Interestingly, however, a disclaimer was often entered to the effect 
that while technical training was necessary, it was net sufficient. 

Eight of the projects made specific mention of the fact that training 
in technical skills was not enough to assure success in educational 
RDD&E activities. The consensus of those comments was that a sensi- 
tivity to the context in which the educational RDD&E activity occurred, 
and an ability to use technical competencies with skill and tact, were 
at least as important, if not more so, than having the skills in the 
first place. 

At 14 of the 20 sites direct reference was made tc the need for 
on-the-job or on-site experience. Many commented that this experience 
was vital. Whenever on-the-job training was mentioned, it was almost 
invariably emphasized as being a high priority need. Of the 14 sites 
making reference to this need, 9 gave it first or second priority among 
the needs mentioned. Also, when on-the-job training was mentioned, 
it was often with the suggestion that college preparation should in- 
clude this element, and that without it preparation for jobs in educa- 
tional RDD&E would be severely limited. Two sites made specific sug- 
gestions to the effect that course work and on-the-job experience 
should occur concurrently in training programs. At a number of sites 
interviewees recommended on-the-job experience during training as a 
means for applying and testing the theories taught in technical and 
academic subjects. It was felt that while technical-academic training 
was essential, on-site experience was necessary if the student was to 
understand fully the nature of what was learned in course work. It 
also seemed clear that trainers were being told that the work of the 
practitioner was already complicated enough without having to teach 
newly trained personnel the basic elements of real-world ’’etiquette", 
and that trainers must share this burden. 

Of the technical skills mentioned the one that was most often 
emphasized was writing. Nine of the 20 sites indicated that writing 
was an essential skill in the performance of their work. While writing 
was almost invariably considered useful, and often essential, few 
respondents had suggestions as to how skills in writing could or should 
be acquired. The concerns most often expressed with respect to this 
skill were that writers be able to communicate clearly to a wide variety 
of audiences and they they avoid the use of jargon or "educationese . ,f 
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A need often expressed by the sites visited was that of a staff 
member being able to work as part of a team. Seven of the sites visited 
made specific reference to this need, and one listed it as being of the 
highest priority. Those citing this need commented on the fact that 
most educational RDD&E activities are carried on by groups rather than 
by individuals, and that for that reason the ability to work within a 
team became a significant skill to have. The suggestion was made that 
dissertations be done as a team rather than by individuals so that those 
participating could master a specific area of specialization (i.e., 
measurement, statistics, design, etc.) and at the same time learn to 
work as part of a group attempting to achieve a specific goal. Associ- 
ated with the notion of the need for teamwork were comments relative to 
staff being able to communicate effectively with each other, being sen- 
sitive to the needs of others, and being able to put the right people 
in the right ,, slots. n 

A set of comments frequently obtained related to the importance of 
a staff understanding and being committed to a common goal or objective. 
This type of comment was prominent in seven of the 20 sites. In each 
case those mentioning this "goal orientation 11 indicated that it was the 
focal point around which all other considerations revolved. It was 
generally felt that placement of staff, use of skills, budgeting, time- 
lines and other activities were a function of the end state being 
sought. It was also felt that technical skills, previous experience, 
and degrees held wfre of little importance in the absence of a commonly 
held, mutually desired goal among staff. In terms of training these 
comments may seem distant if not irrelevant, but in the context of the 
frequently cited need for on-the-job experience it begins to make sense. 
In effect it appears that these people are saying "let's train people 
on-the-job, and let's make clear the goals we're trying to achieve so 
that we can work in this realm." A related comment frequently made was 
that people in the field "need to feel that what they are doing is 
worthwhile . " 

The only other comments made with any degree of frequency in the 
profiles were made with respect to prior experience. In some cases the 
degree of previous experience in the field under xnves tigation or in 
related fields was the sole criterion for employment. Ten of the 
twenty sites visited mentioned experience in educational RDD&E, or in 
related work, as one of the primary sources of training for work on the 
project under consideration. Five of the sites visited recommended ex- 
perience in the same kind of work as that being carried out as requisite 
to the job to be done, while seven listed work in related fields as 
important . 

A number of ether comments, while mentioned by only one or two 
sites, seem worthy of consideration. The lack of reality in training was 
criticized strongly at one site. The notion of learning about educa- 
tional RDD&E within "four walls" was felt to be inadequate to the needs 
of the "real world." This ties in with the frequently mentioned need 
for on-the-job training. Mentioned at least twice was the fact that 
many people working on educational RDD&E projects had their time divided 
between projects. This kind of staffing pattern, which is not uncommon 
for educational RDD&E work, has implications for the latitude and flex- 
ibility of craining needed by people intending to work in the field. 
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Lack of adequate training was mentioned for several specific areas 
including management, finding resources, using consultants, and pro- 
posal writing. 

| A final note must be added with regard to the matter of academic 

i degrees. Surprisingly, in contrast to the data reported in the previous 

> section of the chapter, degrees were rarely mentioned in connection with 

i the necessary requisites for jobs in educational RDD&E. In fact the 

L j comment was made on at least two sites that degrees were irrelevant to 

the work assignments given and the tasks to be performed. Often com- 
ments with respect to the need for degrees conflicted. Some who held 
B.A.'s felt that Masters or Doctorates were needed for the work they 
were doing, while others with Doctorates felt that their work could have 
I been done by someone with a B.A. Perhaps the best summary statements 

| with regard to degrees is a quote from a respondent who said "the Doc- 

torate is totally irrelevant to the work that needs to be done, but it 
is essential for political reasons." 1 What this means apparently is 
^ that the possession of higher level degrees, regardless of the compe- 

tence of the holder, has an influence on the people with whom educa- 
tional RDD&E personnel must work. Degrees appear to be the most quickly 
f accepted credentials that a prospective practitioner can possess. 



Some Suggested Uses of the Volumes 
Reporting the Oregon Studies 



Previous sections of this chapter have addressed implications for 
training that arise from the data presented in Volume I of the Oregon 
Studies, and from Chapter 6 of the profiles contained in Volume IV. 
This section will address the question of the utilization of the other 
volumes of the Studies, Volume II, III, and V, and the utilization of 
the profiles as a whole in the design and operation of a training pro- 
gram for the preparation of personnel to work in RDD&E. 




Volume II: The Literature of RDD&E 

One of the most pressing concerns of the novice upon introduction 
to a new field of study is his need to obtain some grasp of the nature 
and parameters of the field. In terms of the realm of educational 
RDD&E, the beginner's first concerns might revolve around such questions 
as: "What does the field include?", "What defines the context of 
RDD&E?", and "Who are the people associated with present thinking in 
the area?". 

Several possible uses of Volume II exist when considering it as 
a training resource. 

1. As an introduction to the field. 

Prospective professionals who have made the choice to enter 
the fields of research, development, diffusion or evaluation in 
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education could be introduced to some or all of the papers Included In 
the vo J ume as a first exposure to the nature of the career that they 
have chosen. In- reading through the volume they could focus on a 
specific area of interest, c.g., research or development, and gain a 
sense of the issues and concerns that seem to be central to those areas, 
or they could survey the entire set of readings to gain a sense of the 
interrelatedness of the various areas within the domain. 



2. As a preliminary source for exploration in the field. 

Each of the areas, research, development, diffusion, and 
evaluation, is dealt with as an individual topic in some of the articles 
in the volume. Otner articles deal with various combinations of areas. 
The beginning professional who has selected a particular area of con- 
cern for further investigation would do well to seek those articles in 
the volume which relate to his area of concern and study them care- 
fully. This approach weald not only provide him with a relatively com- 
pact and well organized beginning point in his search, but would lead 
to numerous other sources through the references provided for each 
article. 



3. As a supplement to technical professional courses. 

The two parts of the volume provide excellent resource mate- 
rials for any number of courses in professional curricula related to 
the arena of educational RDD&E. Both the design and the content of the 
volume make it a convenient resource for use in a wide range of course 
offerings . 



4. As a source of materials for symposia. 

At an advanced level students or practitioners of educational 
RDD&E can find in the volume numerous articles that merit discussion 
and debate. Using a specific subject, or a combination of articles on 
a "theme", any number of possibilities exist for the advancement of 
conceptual and philosophical thinking within the field. 



5. As a reference manual for on-the-job experience. 

The practitioner of educational RDD&E, whether he be exper- 
ienced or novice, would find Volume II a useful tool as he undertakes 
his own investigations in the field. Articles related to his sphere 
of activity could serve as a check against his own work experience, 
and his experience could help him to more fully understand that which 
is being presented by the author. The blend of his own experience and 
his reading of the literature should serve to enhance his understanding 
of the field and his ability to contribute to it productively. 

These suggestions are of course quite general but they do serve to 
point up a number of potential uses of the volume. The reader, from 
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his own perspective, is encouraged to fine the uses that beat fit the 
needs and purposes he seeks to achieve. 



Volume III: Conceptual Frameworks . 

Volume III contains a set of four papers prepared specifically 
for the Oregon Studies which attempt to "clarify and firm the conceptual 
base that underlies educational research, development, diffusion, and 
evaluation." The volume represents some of the most advanced and delib- 
erate thinking about the field in general that has as yet been done. A 
careful reading of the papers makes evident the fact chat there is as 
yet no unanimously held nor commonly accepted definition for the field, 
nor for the specific areas contained within it. T1 is has interesting 
and significant implications for the training of prospective educational 
RDD&E personnel. 

In "established" disciplines the existence of a set of accepted 
roncepts, definitions, decision rules, rules of evidence and the like 
are taken for granted. Because of the relative youth of educational 
RDD&E as an area of study these elements are by no means clearly in 
evidence* For the prospective educational RDD&E worker this fact is 
of importance, for if he is to be trained, to what view of the disci- 
pline will he be exposed, and what concepts and whose definitions should 
he accept? 

The papers contained in Volume 111 can serve a useful function in 
helping the educational RDD&E trainee come tc grips with such issues, 
though they in no way provide easy answers . They are characterized by 
differences more than they are by similarities. Taken together, how- 
ever, they provide a perspective of the domiin as a whole, and of many 
of the critical issues within it. Using these papers as a base the 
prospective trainee, and the trainer as v.ell, can begin to seek his own 
conceptual frame and definitions. 



Volume IV: The Case Profiles 

By treating each of the 20 case profiles as a reliable descrip- 
tion of "what life is like" within the context of an ongoing educa- 
tional RDD&E project, the profiles as a set could serve a valuable 
training function. They illustrate the nature of the outputs that are 
lii ely to be found within projects, the standards held for those out- 
puts, the work required to produce them, and the knowledges, skills and 
sensitivities drawn upon in carrying out that work. They illustrate 
how the various outputs of work effort link together, the functions 
they serve, and the character they assume* The profiles also portray 
the institutional, political and intellectual contexts within which 
all of the above rest, the organization of people and resources to 
carry out the work required, and the "dynamics" that accompany the 
ever changing mix of all the elements that constitute that which is 
called a project. 

Because the projects studied were selected so as to sample as 
broadly as possible the range of activities found within the domain, 
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the profiles provide an excellent introduction to the various kinds of 
activity likely to be found here. Research, development, diffusion 
and evaluation projects have been profiled; small, middle sized and 
large projects have been profiled; and projects in federally supported 
laboratories and R & D centers, privately supported R & D centers, col- 
leges and universities, and school and state departments have been 
profiled. Taken individually and/or as a whole, the case profiles pre- 
sented in Volume IV constitute the most detailed descriptions of on- 
going educational RDD&E activities available, and as such they should 
serve an invaluable function in introducing students to the field, 
assisting staff who have just entered the field, or assisting project 
directors who need to prepare experienced or inexperienced staff to 
carry out a particular kind of project. 



V ol ume V : The Methodology of the Oregon Studies 

Volume V of the Oregon Studies describes in detail the methodology 
used in gathering .and reporting the data collected in the Oregon 
Studies, As such it represents a major effort at organizing tools and 
techniques from th$ field of educational RDD&E to describe that field. 
From the standpoinc of training it serves as a model, however adequate, 
of one form of output resulting from a major educational RDD&E effort. 

The methodology of the Oregon Studies can be used for training in 
a number of ways. Three are suggested: as an illustrative methodology, 

as a tool for use in replication studies, and as a model from which new 
methodologies can be derived. 



1# As an illustrative methodology. 

The methodology described in Volume V is but one of a number 
of possible methodologies that could have been adopted for the purposes 
of the project. To the interested student it can serve as an example 
of the process by which a methodology is arrived at, and the decisions 
that are involved in its application. One of the advantages of using 
the methodology for purposes of training is the fact that the data that 
result from its application are data that are of substantive value to 
the trainee. 

Given the methodology, and the data that derive from its applica- 
tions, the student of educational RDD&E can make critical analyses 
with respect to the logic of the methodology itself, the facility with 
which it has been applied, and the productiveness and utility of the 
results obtained. Of further use to the student is a study of the 
concepts or language reflected in the methodology. Because of the 
uniqueness of the approach used, a language system had to be developed 
to describe both the methodology and the results. The language devel- 
oped provides a fertile base for analysis and debate on the part of 
both prospective and practicing personnel in the field. 



2. As a basis for replication studies. 



One of the early and valued experiences of most students of 
educational RDD&E is the replication of previously conducted studies. 
The methodology of the Oregon Studies is ideally suited for use in 
replicating small or large parts of the parent study. The results 
obtained from such replications would serve as a valuable check on the 
data presented in the present volume, as well as a valuable learning 
experience . 



3. As a model for the derivation of new methodologies. 

A difficult task for any educational RDD&E practitioner, but 
especially so for the novice, is the development of a methodology that 
reflects new concepts or demands new approaches to measurement. Yet, 
if a science is to progress, new methodologies must always be devel- 
oped. The precision and comprehensiveness of the Oregon Studies 
methodology could well serve as a model for what the development of a 
methodology entails, and what its presentation requires so that others 
can use it reliably. Though Volume V of the Oregon Studies is a rela- 
tively detailed document, and is complex reading in places, the nature 
of the information it holds makes it a significant document for students 
of educational RDD&E to come to know. The beginner in the area of 
educate nal RDD&E would be well advised to have access to Volume V 
of the Oregon Studies as a reference when methodological issues arise. 
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Chapter 15 



IMPLICATIONS FOR TRAINING: DATA DEPENDENCY AND COMPETENCE 



The ultimate measure of the success of a training program is the 
degree to which its "graduates* 1 demonstrate competence in their assump- 
tion of work responsibilities. It is likely that very little disagree- 
ment would be found, either among trainers or practitioners, concerning 
the desirability of the achievement of competence as an end goal for 
trainees. What is likely to result in disagreement is the manner in 
which competence is defined and the process of training by which it is 
achieved. 

The purpose of this chapter is to address the issue of competence 
from the standpoint of training. This falls in line with the charge 
that the Oregon Studies identify the competencies needed by practition- 
ers in the field of educational RDD&E. Two sections are offered, the 
first dealing with an operational definition of competence and the second 
with a training model based on that definition. 



A Focus for Defining Competence 



The definition of the word competence includes such synonymous 
terms as capability, skill, fitness, etc., without reference to the 
'‘something** in which one is to be capable, skilled, or fit. In order 
to understand and make use of the concept of competence, then, this 
something, this context, for extending its definition must be identi- 
fied. Almost any context suffices as long as some information is avail- 
able whieh gives substance to a capability statement, the degree of sub- 
stance is dictated by the amount and specificity of that information. 
Thus, the initial problem is one of specifying a context in such a way 
that substantive meaning is brought to the definition cf competence. 

Familiar contexts in which definitions of competence occur include 
discipline, job role, enablers (e.g., knowledge), etc. Each of these 
may have utility in substantively defining competence depending upon 
the amount and level of specificity of the information one has about 
them. Given a definition of competence within one of these contexts, 
a second problem is raised. This problem is concerned with the prac- 
ticality of training in the competence: is so much involved in the 

definition that training cannot be accomplished in a reasonable time 
frame? Or is so little involved that the meaning of the competence is 
lost relative to the larger scheme of things to which it is designed 
to contribute? Consideration of the reasonableness of utilizing 
selected contexts for defining competence in educational RDD&E appears 
in the following sections. 



Competence and the Discipline of Educational RDD&E 

The first context mentioned in connection with defining competence 



was the "discipline * " A question that could be asked is, "How adequate 
is the concept of "discipline" to serve as the basis for competency 
based training?" The data from the Oregon Studies provide information 
useful in responding to that question. 

Traditional training programs exist to prepare people to work in 
many disciplines. Colleges and universities offer programs in Psychol- 
ogy, History, Art, and Chemistry, for example, so why not educational 
RDD&E? Educational RDD&E, like History, Art, Chemistry and Psychology 
is composed of an almost limitless number of elements, each of which 
encompasses enough "Knowledge" to demand a lifetime of study in and of 
itself. As the data in Chapter 6 of this volume indicate, personnel 
actually working in the field of educational RDD&E received training in 
a variety of "disciplines" including, but not limited to, education, 
psychology, and, peculiarly, engineering. Apparently, if these data are 
to be given their due, educational RDD&E contains within it content 
associated with a wide range of disciplines. In order to use £ disci- 
pline of Educational RDD&E as a context for defining competence it becomes 
necessary to conduct work aimed at identifying the common element within 
the associated disciplines which bring them together. It is also nec- 
essary to identify those areas which make the associated disciplines 
mutually exclusive. Furthermore, the finding that multiple disciplines 
appear within educational RDD&E leads to the realization that even 
efforts to discriminate separate "disciplines" of R, D, D, and E would 
be extremely difficult. Thus, the enormity of the task in isolating 
these interacting variables in order to define a discipline of Educa- 
tional RDD&E in the first place, is such that it is not a practical 
context in which to define competence in a manner relevant to training. 

Competence and Job Role 

The concept of job role appears frequently as an organizer for 
training programs. It is possible to find programs which train people 
to be teachers, mechanics, leatherworkers, psychiatrists and any number 
of other job roles. In several previous chapters within this volume, 
most notably Chapter 12, the concept of job role was discussed in con- 
nection with the data gathering efforts of the Oregon Studies. The 
conclusion presented in those discussions has relevance to the use of 
job role as a focus for competency based training, i.e., in educational 
RDD&E the actual work assumed by personnel holding the same or similar 
job role titles varied dramatically both within and across projects. 

This "slipperiness" in stability associated with job role identification 
in educational RDD&E has precluded, to date, the ability to determine 
those elements which discriminate the similarities and differences among 
various job roles. Such discrimination is a requisite to providing a 
definition of job role suitable for giving stable substance to the def- 
inition of competence in it. Until such time that job analysis efforts 
prove fruitful in stabilizing job roles within educational RDD&E, risks 
are inherent in their use as a context for defining competence at a 
level suitable for training. 

Competence and Enablers 

Many training programs contain elements focused on the acquisition 
of the knowledges, skills and sensitivities considered essential to the 
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execution of job related tasks. Course titles beginning with "Funda- 
mentals of ..." or "Principles of ..." represent examples of training 
efforts at this level. Basic to the development of a training program 
focused at this level are two conditions: 1) the enablers associated 

with work performance are finite and known; and 2) the enablers to be 
trained toward have general utility throughout the discipline. 

The data on enablers contained in Chapter 8 of this volume give 
an indication of the degree to which the two conditions are met. The 
data on knowledges, skills and sensitivities actually required by 
RDD&E personnel in the performance of their work responsibilities indi- 
cated a wide range and limited commonality of enablers across the disci- 
pline, In short the absolute number of enablers required of personnel 
in educational RDD&E combined with the dispersion of those enablers 
across the discipline leaves one without any readily apparent set of 
organizers by which to cluster them. Furthermore, the discussion of 
the data emerging from case profile analyses in Chapter 14 points up 
the inadequacy of "four-wall" possession of enablers when one is faced 
with operating in a line production activity for the first time. Thus, 
caution is suggested in the use of enablers as a focus for defining 
competence . 

Competence and Outputs 

As has been demonstrated, the use of the discipline, job roles, 
or enablers as contexts for building competency based training programs, 
each have risks likely to promote less than desirable results. It is 
likely that any context chosen would have a number of drawbacks. The 
experience of the Oregon Studies staff has, however, led to the selec- 
tion of outputs as the unit of analysis for data collection. It is 
felt that, in the domain of educational RDD&E, the same context can be 
used as a basis for developing training programs geared, toward compe- 
tence . 

The feature of outputs which makes them a unique and desirable 
base for building training programs is their stability (see the Schalock- 
Sell conceptual paper in Volume III of the Oregon Studies). While the 
discipline itself takes on the hue of whatever perspective the observer 
brings to it due to the many elements it contains, while job roles tend 
to slide and shift, and while enablers are virtually infinite in the 
absence of organizers which cluster them, outputs tend to remain stable 
and predictable regardless of where they appear. Granted that the kinds 
of outputs produced by different projects tend to vary, a given output 
(e.g., a report or contract) can be identified as such irrespective of 
the project in which it is produced. A contract is a contract whether 
it is produced in a research project by a top level manager or an evalu- 
ation project by a research assistant. 

The use of outputs as the basis for training offers a number of 
advantages besides stability. 

1. They can be linked to all other dimensions of the domain, 
such as - 

a) context 

b) personnel (job role) 
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(’) other outputs 
d) work requirements. 

2. They provide both the focus of training and the basis for 
evaluation (i.e., output and process standards). 

3. They are generalizable , i.e., personnel producing a specific 
output in a specific setting are likely to find that same 
output or a similar one being produced in other settings. 

4. Failure to successfully produce an output can be "traced 11 back 
to the tasks or enablers involved in its production. 

5. There are a number of dimensions by which outputs can be 
analyzed (i.e., structure, level, character, focus) which 

can lead to a fuller understanding of the context (discipline, 
focus, project) in which they reside. 

If outputs are selected as the focus for building training pro- 
grams, a definition of competence becomes relatively easy to generate 
and program design can be readily developed. The following definition 
of competence uses output as the central organizer: 

COMPETENCE 

The exercise of relevant enablers within a context 
in the performance of tasks designed to produce 
desired outputs having verifiable standards of 
excellence . 

As can be noted by the perceptive reader this definition, while 
it centers around outputs, refers to almost all the data sets Included 
for analysis in the Oregon Studies. In essence this means that it is 
possible to gather and/or assemble data around any aspect of a training 
program based on this definition of competence. 



A Training Model Based on an Output Oriented 
Definition of Competence 



If outputs are used as the level of analysis and focus of a 
competency based training program, the following steps in the develop- 
ment of the program are required or implied. 



1. The context toward which training is aimed must be 
identified. 

2. The outputs desired within the identified context must 
be determined. 

3. The standards, both process and output, for the outputs 
selected must be established. 

4. The expectations of and for individual trainees must be 



determined on the basis of the context, outputs, and 
standards identified . 

In an operational training program it might be expected that the 
trainer would be responsible for identifying one or a number of con- 
texts to serve as the source of program inputs in terms of outputs 
and standards. Given these inputs the trainer could then negotiate 
with a given trainee for the production of a specifiable set of out- 
puts to be produced to the standards established for them. Each of 
the outputs selected could also be further analyzed to determine the 
tasks associated with their production and the enablers required in 
the performance of those tasks. With these sets of data as inputs, 
the trainer could then design instruction in terms of programs, 
modules, packages, experiences, etc. in a way that would maximize the 
opportunity of the trainee to achieve a satisfactory level of compe- 
tence . 

In the design of a competency based program several conditions 
are implied by the output focus. 

1. The program must be field-centered . To be field- centered 
does not necessarily mean that all training must occur 

in a field setting. It does mean, however, that the con- 
text, purpose, and evaluation of the program must derive 
from and culminate in the field. In the absence of an 
arena for the acquisition and demonstration of competence, 
training programs can never be sure that the criteria for 
competence in the field has been met. 

2. The program can be personalized . A unique program can be 
provided each trainee in each unique context. While there 
might be general standards of competence applied to the 
field, the individuality of trainees and of contexts would 
tend to result in programs being personalized to suit idio- 
syncrasies in people and settings within broad limits. As 
many options as possible could be provided the trainee when 
he reaches the point at which he is able to choose his own 
professional goal. Once the trainee has selected the 
setting for which he is preparing he may be trained to 
achieve a relevant set of outputs unique to his talents 
within that specific setting. This requirement emphasizes 
the necessity of establishing a process of negotiation 
designed to assist the trainer and trainee in arriving at 

a "program." 

3. Standards applied to outputs must be clear and public . 
Trainees as well as trainers should be made aware of the 
specific standards applied within contexts and of the 
standards held for similar outputs by "others" in the field. 
The measures that constitute evidence of meeting context 
specific and general standards must be made clear. 

4 . The program can be subject to continuous assessment . 
Assessment of trainees at the level of enablers, tasks and 
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resulting outputs, and assessment of trainee ability to 
assume contextual job roles through the achievement of out- 
puts, is useful, if not essential, to a competency based 
program. Evidence of trainee progress at each level of 
training serves the needs of the trainee in terms of per- 
sonal growth and matriculation, the trainer in terms of 
program modification, and the prospective certifier or em- 
ployer in terms of competencies acquired. 

All the data presented in this volume have given repeated evi- 
dence that educational RDD&E, at least at the present time, is a 
unique domain which calls for unique kinds of training. The model 
suggested here has been suggested specifically by the data obtained in 
the Oregon Studies. While it may have general utility, the model is 
considered to be most appropriate to the specific demands of the do- 
main of educational RDD&E as they exist at this point in time and are 
reflected by the Oregon Studies data. Others who come to understand 
the data from the Oregon Studies will, hopefully, be able to utilize 
the model suggested either to guide training program development or 
as a springboard from which to develop other models for the same data 
base . 



It may well be that it is just as important to give a trainee a 
sense of "role" as it is to help him develop competence in terms of the 
outputs he is able to produce. The sets of outputs selected for train- 
ing may in effect, describe the parameters of a role and thereby help 
to define it in much the same way that the complete Oregon Studies data 
describe the parameters of a domain and help to define it. Using 
output as the unit of analysis in looking at competence may bring 
greater clarity to the understanding of role and the competencies 
associated with it. 

Competence, in the final analysis, is a judgment regardless of 
the level analysis selected. In order for a trainee to "be" competent 
he must be judged so by someone else, e.g., the trainer, certifier, 
employer, etc. The judgments made of a trainee, or of a practitioner 
for that matter, will likely be made on the basis of a variety of 
evidence, including the way the person making the judment "feels" 
about the person being judged. The data from the Oregon Studies sug- 
gest that evidence of the ability of a trainee or practitioner to 
produce outputs to the standards established in a specific context or 
set of contexts is likely to be the most reliable and valid indicator 
of competence. If the outputs selected as the focus of training are 
truly representative of the responsibilities the trainee is eventually 
to assume, evidence as to the trainees facility in producing those 
outputs should provide the "judge" of competence with the major source 
of decision related data. 
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Chapter 16 



RECOMMENDATIONS 



It is customary for an expeditionary force to make recommendations 
to its sponsoring agency upon return. Since the Oregon Studies have been 
cast in the light of an exploratory or expeditionary force it would seem 
that the custom should be maintained. Accordingly, the present chapter con- 
tains a series of recommendations to the agency that sponsored the Oregon 
Studies as to the steps that need to be taken next to gain full advantage 
of what has been accomplished thus far. The recommendations reflect an 
effort to mesh the biases of the staff of the Oregon Studies, the knowledge 
and sensitivities gained as a consequence of the Studies, and the set of 
concerns that prompted the commissioning of the Oregon Studies initially. 
Hopefully the recommendations that are made reflect accurately what has been 
learned as a consequence of the Studies, the relevance of that which has been 
learned to the concerns of the sponsor, and the urgency of the steps that 
now need to be taken. 

In studying the recommendations the reader should be aware of the mul- 
tiple frames of reference within which they have emerged. Four frameworks 
are critical in this regard: (a) the perception that the sponsors* frame 

of reference was and still is primarily that of personnel development, but 
that central within that frame is the desire to establish a data dependent 
model of training and the belief that the theory and data generated in support 
of training has utility within the field of educational RDD&E generally; 

(b.) the charge of the Oregon Studies to further the aims of the sponsor by 
first developing the conceptual and methodological tools needed to establish 
a strong data base for training, and then applying those tools to the develop- 
ment of an initial, and admittedly limited, data base for training; ^ (c) the 
decision made early in the Oregon Studies that the aims of the sponsor could 
be served best by viewing the Studies as basic science activities rather than 
a set of activities aimed at having immediate and directly translatable impact 
upon the design and operation of training programs; and (d) the perception 
of the staff of the Oregon Studies, as well as the advisory body to the 
Studies, that the basic challenge of the Studies was no less than laying the 
groundwork for a discipline of educational RDD&E. The task of providing a 
first approximation to the conceptual structure of educational RDD&E, of 
providing a reliable methodology for the empirical investigation of that 
structure, and of providing a^ first set of empirical observations that tested 
the fit of the proposed conceptual frame with the realities that it was in- 
tended to map, was seen as a task aimed at establishing the cornerstones of 
a discipline. As stated in Chapter 1: 



In the design of the Oregon Studies there was a deliberate decision to 
sacrifice data yield for conceptual and methodological development (see 
Chapter 1 for a discussion of the rationale underlying this decision). 



"Just as the journey of Lewis & Clark opened the western part 
of the United States to exploration, development, and com- 
merce, and the explorations of Cousteau and the Apollo program 
have opened the ocean and the solar system to similar possi- 
bilities, the Oregon Studies were designed to turn the tool9 
of educational RDD&E upon themselves and so open the field to 
exploration and development in service of American education. 
In the judgment of those who have been most deeply involved in 
the project the perception of the challenge as anything less 
would have been to miss the point of the whole effort." 

(p. 15) 



The recommendations that follow are a result of the interaction of these 
multiple frames of reference with that which was experienced in the explora- 
tory expedition that was the Oregon Studies. In recognition that all things 
cannot be done at once, an effort has been made to sort the recommendations 
into those that need to be attended to most urgently and those which can be 
postponed or which follow logically from the first wave of recommended 
follow-on activities. No attempt has been made to accompany the recommenda- 
tions with suggestions as to how the activities proposed are to be implemented. 



First Stage Follow-on Activities 
(July 1, 1972 - June 30, 1973) 



Recommendation 1. 



Recycle the conceptual effort started in the Oregon Studies, utilizing 
the conceptual and empirical base established through the Oregon and AERA 
Studies as a point of departure . 

The conceptual papers and critiques presented in Volume III in the series 
of volumes reporting the Oregon Studies represent the results of a major 
effort to bring about conceptual clarity within the field of educational RDD&E. 
As such the volume represents an excellent base for further conceptual effort. 
Individually, however, the frameworks outlined in the volume are far from 
completed statements, and collectively they signal that the conceptual task 
that confronts the field as a whole has only begun to be approached. Given 
the criticalness of conceptual clarification to any discipline or field of 
endeavor, it seems essential that the task of conceptual development be 
reactivated immediately. The fact that there is now a beginning data base 
against which to reflect that effort is seen as an encouraging condition. 

Recommendation 2 . 

Commission a series of interpretive papers that will spell out the utility 
of that which has emerged from the Oregon and AERA Studies for a variety of 
targeted audiences . 

By casting the Oregon Studies, and to some extent the AERA Studies, 
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in the role of basic science efforts, the form in which they have been 
reported tends to serve the interests of scientists better than the inter- 
ests of practitioners. The purpose of Recommendation 2 Is to begin the 
process of analyzing these reports from the point of view of the utility 
they might have for various consumer groups, and to begin the task of 
translating the reports into a format that permits their utility to be 
recognized and acted upon. In thinking about potential consumer groups 
for the information contained in the reports, the following come to mind: 
the directors and staff of personnel development programs; the directors 
of and staff engaged in ongoing RDD&E activities; the administrators of 
institutions and agencies responsible for the conduct of educational RDD&E 
activities; the administrators of institutions and agencies that work 
cooperatively with educational RDD&E personnel; and persons responsible 
for the development of policy in relation to all such activities. Obviously, 
the form and substance of the interpretive papers would need to vary accord- 
ing to the consumer group being addressed. 

Recommendation 3. 



Recycle the training model development effort that was initiated as a 
companion activity to the Oregon and AERA Studies, using the models that 
have been developed, the experience that has accrued in implementing those 
models, and the data that have emerged from the Oregon and AERA Studies as 
a point of departure . 

Just as the conceptual papers developed within the Oregon Studies rep- 
resent only a first step towards the conceptual task that needs to be 
completed within the field, the data dependent models that have been developed 
thus far for the preparation of personnel within the field represents only 
a first step in the overall task of model building. The models are as 
yet incomplete and they have only begun to be tested. Given the significance 
of strong personnel development programs to a professional area such as 
educational RDD&E, and the potential generalizability of a data dependent 
model of training to other areas of professional preparation, it would seem 
that the model building effort should be reactivated immediately. The 
availability of the models developed during the course of the initial 
effort, the experience that has been gained in their implementation, the 
data base that now exists that did not exist at the time of the initial 
effort, and the availability of comparable models from other areas of 
professional education should facilitate the task considerably. 

Recomm en dation 4 . 

Assign a task force to review the data generated within the Oregon 
and AERA Studies to determine the additional analyses that should be run 
on it, and the nature and extent of the data that should be added to it* 

Because of time and resource constraints the analyses carried out 
and reported in the Oregon Studies represents only a partial set of the 
analyses that could be undertaken with the data available. This is prob- 
ably the case as well for the AERA Studies. Given the large investment of 
resources in the collection of these two sets of data, and the potential 
utility that resides within them, it would seem wise to have them analyzed 
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as fully as possible. The preparation of the interpretive papers that 
have been recommended would serve a most useful guidc^on function in this 
respect. Those papers should also serve a useful guidance function in 
determining additional data needs, for in the course of their preparation 
both the needs of a variety of persons who would like to use such data 
and the adequacy of the data to meet those needs will be determined. 

Recommendation b . 

Representatives from the various consumer groups to be served by rny 
or all of the activities recommended thus far should be party to those 
activities . 

It seems desirable to include the representatives of the various 
consumer groups on three counts, (a) their interest in and support of 
the implementation effort would facilitate the production of relevant 
outcomes, (b) insuring the relevance of those outcomes would be in the 
best interests of all concerned, and (c) their participation is in 
keeping with those strategies found essential to serving the purposes 
of development, diffusion, and evaluation. 



Second Stage Follow-on Activities 
(July 1, 1973 - June 30, 1974) 



Recommendations : 

6. Support some or all of the extended data analyses that would likely 
be proposed as a result of following Recommendation 4. 

7. Support some or all of the new data collection efforts that would 
likely be proposed as a result of following Recommendations 1, 2, 

3, and 4. 

8. Support hypothesis testing research into the nature of educational 
RDD&E, and/or its interaction with on-line educational practice 
(such research would likely follow from the conceptual activity 
that would result if Recommendation 1 were pursued). 

9. Design and implement an "inservice" training program for the 
directors of training programs that would keep them abreast of 

the model development activity that would result if Recommendation 
3 were pursued. 

10. Support the preparation of instructional materials that translate 
to a usable form that which has emerged from the 0::egon and AERA 
Studies (the need for such materials would likely follow from the 
preparation of the interpretative papers that would result if 
Recommendation 2 were pursued). 



Continuing Activities 
(July 1, 1974 - -) 



Recommends t ions : 

11. Continue to support the development of the data base needed for 
training and other applied activities. 

12. Continue to support hypothesis testing research on the nature 
of educational RDD&E, and/or its linkage to the practice of 
education. 

13. Continue to support conceptual efforts so long as those efforts 
interact with an accumulating data base. 

14. Continue to support model building efforts so long as those efforts 
interact with an accumulating conceptual and data base, and 
comparable model building efforts in related fields. 

15. Continue to commission papers that interpret all of the above to 
the various consumer groups that find meaning in them. 




* 
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EPILOGUE 



The Parable of the Roads^ 



’’Would you tell me, please, which way 
I ought to go from here?" (asked Alice.) 

"That depends a good deal on where you 
want to get to," said the Cat. 




The exchange between Alice and the Cat is most fitting as an 
introduction to the last section of the paper, for in the writers' 
judgment, educational RDD&E is at a cross roads. From our view there 
are three reasonably well defined roads which can be followed, though 
none promises a particularly smooth ride. 

The firs, is the road that has characterized educational RDD&E 
historically. To a large extent this is a road dominated by classical 
research design and classical research interests in education, for 
example, acquisition and retention studies, transfer studies, teacher 
characteristics studies, and teacher behavior studies. Evaluation, 
development, and diffusion, as these concepts have been discussed in 
the present paper, are largely missing. In the light cf the present 
day this is a relatively narrow road and one that is marred by potholes, 
blind corners, and an almost nonexistent sign post system alerting the 
traveler as to where he is going. 

The second road that is open to travel by educational RDD&E per- 
sonnel is one that has been opened only recently and one that is not 
yet completed. This is the road that takes the form of a super-highway 
begun less than a decade ago and has lanes accommodating not only re- 
search, but development, diffusion, and evaluation, as well. And, each 
lane of the super-highway is sufficiently wide and well-surfaced so as 
to accommodate new, high speed, high powered, roomy vehicles. Exemplary 
efforts along th “tse new lanes are such endeavors as "county agents" 
for encouraging and demonst. iting innovative educational methodologies, 
the vast ERIC dissemination system, educational accountability and per- 
formance contracts, systematic derivation and structuring of relevant 
student performance objectives, and development of major instructional 
packages for increased management of the total learning situation for 
individual students. The problem with this road is that it is still 
under construction; it has not as yet been surfaced; and, only the first 
few miles have been established. As a consequence there are signs all 
along the way that read "Proceed with Caution," "Work in Progress," 

"Detour," and "Stop! New Road Under Construction." There are sign 
posts along the way that tell the traveler where the road has come from, 
and where it ultimately hopes to go, but confidence in che likelihood 
that it will even reach its destination is a matter of faith. 

ItTuT "Parable of the Roads" appears as the last section within the Schalock- 
Sell conceptual paper in Volume III of the Oregon Studies, is quoted 

here in its entirety. 
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From the writers' view this situation is not disappointing. The 
Oregon Studies alone have generated considerable amounts of new data 
about the nature of RDD&E activities, more than can be accommodated 
fully by a training program in its initial year or two of operation. 
Hopefully, additional studies will soon be undertaken so that, as the 
data generated by the present studies are internalized, the next level 
of data will become available. This assumes, of course, that there will 
be some who will travel the access road that leads to empirical research 
and development with respect to educational RDD&E, a hope that is contin- 
gent upon commitments of the keepers of the road. 

Notes to the Keepers of the Roads 



Like the personnel who staff RDD&E activities, those who determine 
policy about such matters and control funds that support such activities 
have come to the same crossroads. And, as in the case of RDD&E personnel, 
It Is the writers' belief that they should support work on all three 
roads. The basis for this belief is not the simple, blind assumption 
that good things will come from the indiscriminate support of all kinds 
of activities. It rests Instead on the kinds of arguments that have been 
outlined in the paragraphs above, and the belief that the essentially 
yet-untraveled road represents a significant new option for policy-makers 
to support. 

It is also our belief that the new option will be an attractive one 
at the policy-setting level, for in it there is promise of sharpening 
the tools with which to attack the problems encountered in the nation's 
schools. The Investment of what is now billions of dollars in RDD&E 
activities — through Title III of the ESEA legislation, through the Coop- 
erative Research Act, through the decade of support for the National 
Center for Educational Research and Development, the Bureau of Research, 
and the like — must be seen as evidence concerning the faith of the Con- 
gress and others in the power of the activities about which this paper 
is written. But the perpetual disappointment in the payoff of such 
activities in the little good that can be traced directly to them, 
must be having its effects upon the strength of that commitment. Because 
the Investment in the empirical investigation of RDD&E would be relatively 
small compared to the overall budget for such activities within the context 
of education, because the time span needed for such research is relatively 
short, and because the potential payoff is so high, extended work upon 
the access road and its support systems should be undertaken. Just as 
the microscope and the telescope and the laser beam and the cloud chamber 
have opened vast new arenas of research and development in other fields, 
with all of their attendant consequences for the social good, the develop- 
ment of the methodology of RDD&E could have similar consequences. 

The keepers of the roads are, after all, the ones responsible for 
the quality of the road system that exists. If good roads are wanted 
or needed, then good technology must be established for their creation. 

The empirical investigation of the domain of educational RDD&E would 
develop such technology. 
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The third road discernible at the cross roads is an access 
road that parallels the freeway. Its ultimate purpose is to provide 
the support systems needed to plot the course of the freeway, build it, 
maJntain it, service its travelers, and develop new capabilities at road 
building. It leads to the tool shops, the research laboratories, and 
the testing centers of the road builders. It also leads to centers 
that prepare personnel to man the roads, or to regenerate or re- 
freshen those who have been on the road for a long while. 

Unfortunately for the traveler, this is a newly commissioned road, 
and thus far only its pathway has been marked. ^ The traveler is not 
at all sure where the road leads or what the tool shops, research lab- 
oratories, testing stations, or personnel training centers will look 
like. Moreover, there is no assurance that the road will ever be 
opened, or that the support services it is supposed to give access to 
will ever be established. The keepers of the roads are not at all 
sure that an access road, and the support systems it is designed to 
provide, represent good investments. After all, the concept of the 
freeway emerged without such "fancy extras ;" and, it's even built with- 
out their assistance. Why should precious resources be spent on that 
which isn't absolutely necessary, and which may not be needed at all? 

When standing at the r.ross roads which of the three options 
available should the RDD&E traveler pursue? Should he take the well 
traveled road of research, taking particular care to avoid the potholes, 
approach the b! ind curves with caution, and take extra care to be sure 
that the road is leading him to the end he wishes to reach? Or, should 
he enter the superhighway vh?**e, even with all the "Proceed With Caution" 
and "Road Under Coi.struc t ion" signs, the other travelers are pressing 
their new machines at such a pace that there is danger to those who have 
a smaller machine, or who wish to drive more slowly? Even with the 
danger of being run over or shoved out of the way and with all the 
noise, the dust, and the confusion of the big machines moving down all 
four lanes, the superhighway still has some attractions. It looks like 
it’s going in the direction a lot of RDD6E personnel wish to go, and it 
looks like it would let one get there relatively fast. And, once the 
road is surfaced, it should be relatively smooth traveling. Such 
attractions are hard to turn down. 
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The commission for the access road and some of its support 
systems came less than two years ago with the decision on the part of 
the Office of Personnel Development and Training within the National 
Center for Educational Research and Development, U.S. Office of Educa- 
tion, to establish an empirical base for their training programs. Some 
advance survey parties such as Clark and Hopkins (1969), Sieber and 
Lazarsfeld (1966), and Cronbach and Suppes (1969), had studied the advis- 
ability and feasibility of creating such a road. But, it was not until 
the funding of the AERA Task Force Studies, the Oregon Studies, and the 
Training Design Projects that well financed survey parties and planning 
studies were established. The results of the Training Design Studies 
are in and three personnel training centers have been established. The 
results of the survey parties are only now being completed so the future 
of the access road is still uncertain. 
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What are the attractions of the road that leads to the information 
and technology base that permits the improvement of the first two 
roads or the building of better roads in the future? It leads in a 
sense to the source of that which makes more hopeful the prospects of 
traveling either of the other two roads. All that equipment looks 
impressive — it should help — but too few have used it and many lack 
confidence to climb into the operator’s seat. 

So what road should the RDD&E traveler choose? Fortunately , 
while one man has to make one choice , the discipline of educational 
RDD&E can choose all three , and pursue them simultaneously . It is 
the writers’ belief that this is the strategy which should be followed . 
The rationale underlying this position, some recommendations as to 
steps to take along ea rh way, and some implications of the present paper 
for those steps are outlined in the paragraphs which follow. The paper 
closes with some general comments on the applicability of the present 
paper to personnel training and policy setting. 

Following an Improved Old Road 

The case has been built sufficiently well and sufficiently often 
for the necessity of a solid knowledge base for any of man’s endeavors, 
so that argument need not be repeated here. What does seem to be worth 
saying is what the conceptual frameworks or the theoretical structures 
must be in order to develop a strong knowledge base. The constructs or 
concepts that constitute those frameworks must be well explicated and 
critically tested for their utility. And, advances in design and 
methodology must keep pace with the advances in theory so that the 
power of such frameworks can be fully utilized. 

Fortunately, the kind of conceptual base being called for is 
beginning to emerge in education. The work on aptitude treatment 
interaction, the concept of mathemegenic research (Rothkopf, 1970; 

Frase, 1970), the concept of learning environments (Barker, 1968; 
Mitchell, 1969), and the concept of a teacher as the manager of 
learning environments (Tyler, 1950; Schalock, Kersh, and Horyna, 1970a; 
Schalock, 1971) are all indicative of the kind of change envisioned if 
research is to provide the knowledge base that has utility to other 
researchers or to practitioners in the field. 

Joining with the Builders on the New Road 

Given the need for, the power of, and the demonstrated potential 
of the tools of systematic development, evaluation, and diffusion, there 
would seem to be no other choice but to join forces with the engineers 
of these activities. New ground is being broken in e*ch of these areas, 
some wisdom is accumulating, methodologies are being devised and tested, 
and enough outputs are emerging from them that have enough value to 
enough people that a wide-spread base of support and enthusiasm for them 
is being engendered. As pointed out in the present paper, however, 
there is sufficient confusion and uncertainty in all three of these 
areas that their effectiveness at this point in time is limited. It is 
also probable that it will continue to be limited until our knowledge 
about these activities is significantly increased. A major purpose 
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of the present paper was to provide a conceptual framework that would 
facilitate the extension of that knowledge base. 

Utilizing the Modernized Tools and Resources 

In the writers 1 view the critical road to be followed in the immed- 
iate future is tte road that leads to the increased understanding of the 
domain of educational RDD&E. The case has been argued in the present 
paper that the activities that comprise this domain constitute the primary 
problem solving tools available to the field of education, and that to 
employ them well they must be more fully understood than they are at present. 
It is also argued that a necessary way to increase our understanding 
about such tools is to submit them to empirical investigation. The major 
thrust of the present paper is a conceptual framework that defines and 
relates the conceptual structure of the domain in such a way that it 
could become subject to empirical investigation. Given the availability 
of a framework (assuming that it is adequate to the task) , the critical 
need is for its empirical referencing. A great deal of time should not 
be wasted before learning to use the newer tools and resources emerging. 

Personnel Training Centers 

The context within which the present investigations were commissioned 
was one of personnel training. Largely because of dissatisfactions with 
the old road and the manpower needs of the builders of the superhighway, 
the move to obtain empirical data on the domain of educational RDD&E was 
undertaken. New training programs have been established^ and are ob- 
stensibly awaiting the data from the survey parties to help guide the 
substantive development of their programs. The question that must now 
be considered is one of the utility of the data that have been collected. 

Without question, the data from the empirical studies will have 
some utility for the new and existing training programs for educational 
RDD&E manpower. While speaking now only for the Oregon Studies, data 
will be forthcoming on all dimensions of the conceptual sets provided 
in the previous section of the paper. While these data are primarily 
descriptive and parameter testing, rather than normative or hypothesis 
testing, they will do much to identify the nature of RDD&E activities 
as they appear within education at this point in time. As such, this 
should provide necessary sensitivities to the designers of training pro- 
grams, even though they will not provide definitive answers to sub- 
stantive questions about the field.* * This is not an unexpected circum- 
stance, for the decision was made early in the Oregon Studies to sacri- 
fice data yield for methodological development. 



3 Three experimental training programs have been established as of 
this writing: The Far West Laboratory Consortium which focuses on the 

preparation of developers and diffusers; the Learning Research and Devel- 
opment Center in Pittsburgh which focuses on the preparation of instruc- 
tional systems designers; and the Consortium of the Ohio State University 
which focuses upon the preparation of field -centered evaluators. 

* A number of training, staffing, and organizational Issues were 
raised during the course of the Oregon Studies Project. Volume I presents 
these Issues in light of the data which have been collected and analyzed. 



GLOSSARY 



ADOPTION 



1. the taking of and using as one’s own. 2. in present usage, 
refers to the acceptance for use of KNOWLEDGE, INFORMATION, 
and/or TECHNOLOGY . 



CLUSTER 1. a number of different things grouped together on the basis 
of more general qualities shared among them. 2. in present 
usage, refers to a basis for grouping and classifying large 
numbers of specific OUTPUTS and TASKS, such groupings referred 
to as clus ter categories . 

CHARACTER 1. the distinctive kind or sort that a thing is. 2 . in 
present usage, refers to a classification given an OUTPUT 
that is distinguishable as an instance of KNOWLEDGE, TECHNOLOGY, 
IMPLEMENTATION, or INFORMATION. 

COMPONENT 1. a part or integral ingredient in a whole. 2. in present 
usage, a classification given an OUTPUT having such relation- 
ship to a FOCAL OUTPUT. See FOCAL and FACILITATING. 

CONDITION 1. manner, circumstance, or state of being. 2. in present 
usage, a STRUCTURE classification given an OUTPUT which 
represents an instance of a desired circumstance expected 
to endure within the life of a PROJECT, or beyond the PROJECT 
as a result of it. 



CONTEXT 1. the situation, background, and environment within which 

a specified set of focused activities occurs. 2. in present 
usage, ’’focused activities'* refers to PROJECT, and **situat ion . . . 
environment'* to its FOCUS, SETTING, SIZE, CONTENT. 

DEVELOPMENT 1. a working out in detail or expansion of (a technique). 

2. in present usage, refers to: (a) engaging in a problem- 
solving strategy designed to produce reliable TECHNOLOGY, 
that is, procedures, materials, hardware, and organizational 
frameworks that have a known degree of success in bringing 
about a particular outcome or in performing a defined OPERA- 
TION; (b) a FOCUS classification given a PROJECT. 

DIFFUSION 1. the conveying of something from one person, place, or 

situation to another. 2. in present usage, refers to: (a) 

a problem-solving strategy designed to bring about the IMPLE- 
MENTATION of generalizab le KNOWLEDGE, a reliable TECHNOLOGY, 
or trustworthy INFORMATION (as used here diffusion incorpor- 
ates the concepts of DISSEMINATION, ADOPTION, and UTILIZATION); 
(b) a FOCUS classification given a PROJECT. 



DISSEMINATION 1. scattering far and wide. 2. in present usage, refers 
to the distribution of KNOWLEDGE, INFORMATION, and/or TECHNOL- 
OGY to a targeted population. 



•iUUCAT I ONAL KDD&F. a coordinated set of prob Lem-soi vi ng strategies 



ENABLER 


designed to produce OUTPUTS that can be Judged as to their 
quality and their contribution to the solution of educational 
problems . 

1. that which makes able, provides with means, power or 
authority. 2. in present usage, it refers to those KNOWL- 
EDGES, SKILLS, and SENSITIVITIES needed to produce a particu- 
lar OUTPUT. 



EVALUAT ION 1. finding the values or amounts of (a phenomenon). 2. in 



EVENT 


present usage, refers to: (a) engaging in a problem-solving 

strategy designed to produce trustworthy INFORMATION regarding 
a phenomenon which occurs in e context or environment over 
which the user expects to exercise influence or about which 
he expects to make decisions; (b) a FOCUS classification given 
a PROJECT. 

1. happening or occurrence. 2. a STRUCTURE classification 
given an OUTPUT which represents an instance of occurrence 
of an observable transaction or set of behaviors. 



FACILITATING 1. making easier or assisting the performance or accom- 





plishiuent of. 2 . in present usage, a classification given an 
OUTPUT having such relationship to and which is not an instance 
of a COMPONENT or FOCAL OUTPUT. See COMPONENT and FOCAL. 


FOCAL 


1. of or pertaining to the center of activity or attention as 
a matter of contractual (implied or otherwise) obligation. 

2. in present usage, a classification given an OUTPUT having 
such relationship to the outcomes of the efforts of a PROJECT. 
See COMPONENT and FACILITATING. 


FOCUS 


1. concentration: as, a focus on a kind of strategy. 2. in 

present usage, refers to a classification given a PROJECT hav- 
ing as the primary concentration of problem-solving work ef- 
fort a strategy characteristic of either RESEARCH, DEVELOPMENT, 
DIFFUSION, or EVALUATION. 


FUNCTION 


1. a special duty or service rendered by a person or thing in 
the course of work or activity. 2. in present usage, refers 
only to a classification given an OUTPUT in accordance with 
the POLICY (setting), MANAGEMENT, or PRODUCTION service it ren- 
ders to the course of PROJECT work. 



IMPLEMENTATION 1. the carrying into effect and using of. 2. in present 



, \ 

INDEX 


usage: (a) a CHARACTER classification given an OUTPUT which 

represents an instance of the ADOPTION and UTILIZATION of 
KNOWLEDGE, TECHNOLOGY, and/or INFORMATION; hence, (b) an OUTPUT 
of DIFFUSION. 

1. a list describing the items of a collection. 2. presently 
used in relation to an OUTPUT index, a listing of the OUTPUTS 
of a PROJECT. 
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INFORMATION 1. trustworthy facts or data descriptive of something with- 
out reference to interpretive or value judgments relating to 
that description. 2. in present usage: (a) a CHARACTER 

classification given an OUTPUT which represents an instance of 
such facts or data; hence, (b) an OUTPUT of EVALUATION. 

KNOWLEDGE 1. empirically verifiable ideas inferred from facts or data 

which add to that which is known. 2. in present usage: (a) a 

STRUCTURE classification given an ENABLER which represents an 
instance of that which is to be known as a requisite to accom- 
plishment of an OUTPUT; (b) a CHARACTER classification given 
an OUTPUT which represents an instance of such newly inferred 
and generalizable ideas; hence, (c) an OUTPUT of RESEARCH. 

LEVEL 1. a position within a dependent structure. 2. in present 

usage, level refers to one of three positions in the dependent 
ordering of OUTPUTS, i.e., FOCAL, COMPONENT, FACILITATING. 

MANAGEMENT 1. the act or art of handling, orchestrating, and directing. 

2. in present usage, a FUNCTION classification given an OUTPUT 
which orchestrates the resources (time, personnel, materials, 
space, INFORMATION) available to a PROJECT in the realization 
of outcomes expected from it, or is a report of that orches- 
t ration. 

MAP 1. a drawing or other representation of the relationships of. 

2. in present usage, (a) a schematic ordering of the OUTPUTS 
of a PROJECT in accordance with their interdependent relation- 
ships (OUTPUT map ) ; (b) a schematic ordering of the organiza- 
tional structures which exist within the CONTEXT of a PROJECT 
(Contextual map ) . 

OPERATIONS 1. any movement or series of movements made in fulfilling a 
purpose. 2. in present usage, refers to movements within a 
PROJECT in terms of the TASKS performed in the generation of 
the OUTPUTS for which the PROJECT is responsible. 

OUTPUT 1. something manufactured, created, or produced. 2. in present 
usage, refers to: (a) an identifiable outcome of targeted 

work activity which contributes to the realization of PROJECT 
goals; (b) a STRUCTURE classification given STANDARDS which 
serve as criteria for judging the adequacy of an outcome of 
targeted work activity. 

POLICY 1. any governing principle or guideline for a course of action. 

2. in present usage, a FUNCTION classification given an OUTPUT 
which establishes the principles or guidelines to be attended 
by a PROJECT, such classification frequently referred to as 
policy setting. 

PRIMARY 1. fundamental, elemental. 2. in present usage, refers to an 
initial categorization of interview data in which the category 
retains the fundamental and essential elements of each specific 
datum; hence, primary categories of OUTPUTS and TASKS. 



PROCESS 


1. a continuing procedure generally composed of many steps 
and/or involving many changes. 2. in present usage, a term 
combined with the term OPERATIONS as a STRUCTURE classifica- 
tion given STANDARDS which serve as criteria for judging the 
adequacy of procedures associated with specified OUTPUTS . 


PRODUCT 


1. something tangible or "hard" produced as an outcome of work 
effort. 2. in present usage, a STRUCTURE classification given 
an OUTPUT which represents an instance of a tangible outcome, 
concrete in form, and transportable at a given point in time. 



PRODUCTION 1. the act of fabricating or creating. 2. in present usage, 





a FUNCTION classification given those OUTPUTS which build 
directly on the fabrication objectives of a PROJECT. 


PROJECT 


a formally recognized, funded and directed effort aimed at 
achieving one or more specified ends that have their defini- 
tion in educational RESEARCH, DEVELOPMENT, DIFFUSION, and 
EVALUATION. 


RESEARCH 


1. studious inquiry, examination, investigation, or experi- 
mentation aimed at the generation of KNOWLEDGE. 2. in present 
usage, refers to: (a) engaging in a problem-solving strategy 

designed to produce ideas, principles, theories, and laws 
that can be shown to be generalizable through empirical veri- 
fication; (b) a FOCUS classification given a PROJECT. 



SENSITIVITY 1. a quality of having perceptiveness and responsiveness. 





2. in present usage, a STRUCTURE classification given an 
ENABLER which represents an instance of a specific percep- 
tiveness and responsiveness considered requisite to accom- 
plishment of an OUTPUT. 


SETTING 


1. environment, background, or surroundings of a place. 

2. in present usage, setting refers to a classification given 
a PROJECT residing essentially in a college or university, 

R&D agency, private firm, or public school environment. 


SIZE 


1. extent, scope, or magnitude of a thing. 2. in present 
usage, refers to the current fiscal year funding and number 
of professional staff of a PROJECT, taken independently and 
in combination. 


SKILL 


1. ability, proficiency, or expertness in accomplishing some- 
thing. 2. in present usage, a STRUCTURE classification given 
an ENABLER which represents an instance of a specific ability 
considered requisite to accomplishment of an OUTPUT. 


STANDARD 


1. a grade of excellence or attainment regarded to be a meas- 
ure of the adequacy of something. 2. in present usage, a 
specific criterion applied to, excellence expected of, or a 
criterion by which judgments of adequacy are made about OUT- 
PUTS and PROCESS/OPERATIONS. 




STRUCTURE 1. an orderly arrangement of the parts of a whole. 2. in 
present usage, refers to the constructs which identify the 
parts of each of several wholes, as in s truct ure of OUTPUTS 
(PRODUCTS, EVENTS, CONDITIONS), structure of STANDARDS (OUT- 
PUT , PROCESS/OPERATIONS), structure of ENABLERS (KNOWLEDGES, 
SKILLS, SENSITIVITIES). 

TASK 1. any discrete piece or unit of work. 2. in present usage, 

a discrete unit of work performed among other similarly 
targeted units in producing or achieving a specific OUTPUT 
to the STANDARDS set for it. 

TECHNOLOGY 1. a technical method of or framework for achieving a 

practical purpose. 2. in present usage: (a) a CHARACTER 

classification given an OUTPUT which represents an instance 
of a reliable strategy, procedure, hardware, or set of 
materials designed to bring about a particular outcome or to 
perform a defined OPERATION; hence, (b) an OUTPUT of 
DEVELOPMENT. 

UTILIZATION 1. the making use of. 2. in present usage, refers to 

the making use of KNOWLEDGE, TECHNOLOGY, and/or INFORMATION. 
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the Number of Outputs 
Project Focus 







Primary Output Category 




Project 


Focus 
















Output 


Set 












Totals 


FGHI 


Products 


Res 


Dev 


Dif 


Eval 


(20) 


Coding 




(5) 


(7) 


(5) 


(3) 




0158 


Evaluation Report 


1 


1 


12 


10 


24 


0013 


Project Time Line/Schedule 


4 


6 


7 


3 


20 


0187 


Descriptive Data Rests, Subject 


14 


1 


3 


1 


19 


0006 


Terminal Report 


6 


3 


4 


3 


16 


0082 


Questionnaire 


2 


5 


6 


2 


15 


0040 


Description of Program/Project 


1 


3 


8 


2 


14 


0089 


Status Condition Report 


1 


2 


5 


6 


14 


0095 


Description/ Specs for Output 


0 


12 


2 


0 


14 


0218 


Quality Control Schema 


0 


14 


0 


0 


14 


0227 


Compendium: Instr Objvs/Meas 


0 


14 


0 


0 


14 


0249 


Staff Development Tmg Unit 


0 


2 


12 


0 


14 


0099 


Progress Report 


4 


5 


3 


1 


13 


0007 


Proposal 


2 


6 


1 


1 


10 


0250 


Pupil Svc Personnel Trng Unit 


0 


0 


1C 


0 


10 


0046 


List of Field Settings 


1 


4 


4 


0 


9 


0061 


Observation System 


1 


1 


2 


5 


9 


0256 


Field Test Plan 


0 


5 


4 


0 


9 


0220 


Data Analysis Procedures 


4 


3 


0 


1 


8 


0281 


Evaluation Plan/Design 


0 


6 


1 


1 


8 


0053 


Abstracted Descript: Programs 


0 


2 


5 


0 


7 


0081 


Interview Guide 


2 


4 


0 


1 


7 


0117 


Program (within project) Plan 


0 


1 


2 


4 


7 


0144 


Financial Report 


1 


0 


6 


0 


7 


0164 


Diffusion Manual/Guidelines 


0 


0 


7 


0 


7 


0201 


Professional Report /Monograph 


2 


3 


1 


1 


7 


0217 


Observation Record Form 


0 


5 


2 


0 


7 


0222 


Data Collection Procedures 


1 


3 


2 


1 


7 


0266 


Journal 


0 


0 


7 


0 


7 


0324 


Budget Allocations 


1 


3 


2 


1 


7 


0002 


Research Design 


3 


1 


1 


1 


6 


0034 


Instructional Unit Use Guide 


1 


4 


1 


0 


6 


0055 


Report Specifications 


0 


0 


3 


3 


6 


0108 


Supplementary Instr Materials 


0 


6 


0 


0 


6 


0203 


Learning Activity Books 


0 


0 


6 


0 


6 


0224 


Attitude Meas, Cognitive Value 


0 


4 


0 


2 


6 
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Primary Output Category 


Project Focus 


Output 


Set 






Totals 


FGHI 

Coding 


Products 


Res Dev Dif Eval 
(5) (7) (5) (3) 


(20) 



0230 Technical Report 

0267 Indexes 

0286 Survey Reports 

0017 Terminal Instr Objectives 

0044 Interview Data/Information 



3 0 12 

0 0 6 0 

0 0 2 4 

0 5 0 0 

0 113 



6 

6 

6 

5 

5 



0067 Implementing Plan, I-M Handling 0 4 
0097 Journal Article, Published 2 2 
0135 Television Program Magazine 0 0 
0140 Promotional Phonograph Records 0 0 
0160 Training Film for Educators 0 0 



1 

1 

5 

5 

3 



0 

0 

0 

0 

2 



5 

5 

5 

5 

5 



0169 Diffusion Impact Data 

0188 Normative Data 

0190 Product Review/Critique 

0223 Taxonomy for Learner Outcome 

0228 Compendium: Instr Objectives 



0 

0 

0 

0 

0 



3 
0 

4 

5 
5 



2 

2 

1 

0 

0 



0 

3 

0 

0 

0 



5 

5 

5 

5 

5 



0235 Baseline Data 

0252 Information Brochures 

0261 Formative/Field Test Eval Data 

0001 Prototype Tryout Data 

0031 Criterion- Based Test Item Pool 



3 2 0 0 

0 0 3 2 

0 2 0 3 

13 0 0 

0 4 0 0 



5 

5 

5 

4 

4 



0033 Learner Data File 
0085 Project Operational Plan 

0145 Intraagency Opns Control Form 

0185 Needs Identification Data 
0199 Item Analysis Data 



0 112 
0 4 0 0 

10 3 0 

0 2 11 

0 2 2 0 



4 

4 

4 

4 

4 



0211 Handling Sys: Project Outputs 

0226 Rationale for Objvs and Msmt 

0303 Computer Instr in Computer Use 
0024 Audio Tape Instr Modules 
0038 Research Data 



0 0 4 0 

0 4 0 0 

0 4 0 0 

0 111 
1110 



4 

4 

4 

3 

3 



0042 Selection Criteria: Sites etc. 

0083 PERT Chart 

0118 Formulated Policy 

0146 Contracts for Services 

0154 Specifications for Personnel 



0 0 2 1 

0 3 0 0 

0 0 3 0 

0 0 3 0 

0 0 12 



3 

3 

3 

3 

3 
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Primary Output Category 




Project 


Focus 
















Output 


Set 












Totals 


FGHI 


Products 


Res 


Dev 


Dif 


Eval 


(20) 


Coding 




(5) 


(7) 


(5) 


(3) 




0183 


List of Resource Personnel 


0 


1 


2 


0 


3 


0195 


Social Interaction Data 


1 


0 


1 


1 


3 


0204 


Learning Activity Toys/Games 


0 


0 


3 


0 


3 


0244 


Coded Data 


1 


0 


2 


0 


3 


0279 


Conceptual Framework 


0 


2 


1 


0 


3 


0289 


Directories 


0 


0 


1 


2 


3 


0309 


Worksheets: Proj Documentation 


0 


0 


3 


0 


3 


0332 


Magazine/Newspaper Articles 


0 


0 


3 


0 


3 


0019 


Content Outline, Instructional 


0 


2 


0 


0 


2 


0026 


Computer Asst/Ext Instr Units 


0 


2 


0 


0 


2 


0029 


Site Assessment Criteria 


0 


0 


1 


1 


2 


0047 


A/V Descriptions of Programs 


0 


0 


2 


0 


2 


0049 


Progress Check Test 


0 


1 


0 


1 


2 


0052 


Panel Book Visuals: Audio Tape 


0 


1 


1 


0 


2 


0074 


Specs: Operation Requirements 


0 


2 


0 


0 


2 


0076 


Specs: I-M Handling Sub-Systems 


0 


2 


0 


0 


2 


0078 Major Task Descriptions 


0 


2 


0 


0 


2 


0087 


List of Potential Output Users 


0 


2 


0 


0 


2 


0091 


Performance Specs: I-M System 


0 


2 


0 


0 


2 


0092 i 


Design Alternates for Evaluation 


0 


2 


0 


0 


2 


0096 


Evaluation Checklist 


0 


1 


0 


1 


2 


0104 


Instructional Package, Medium 


0 


2 


0 


0 


2 


0105 


Instructional Package, Small 


0 


2 


0 


0 


2 


0121 


Script, Film 


0 


1 


1 


0 


2 


0129 


Television Program 


0 


0 


2 


0 


2 


0131 


Live Action Film: Reading Prog 


0 


0 


2 


0 


2 


0132 Video Tape, TV Prog Component 


0 


0 


2 


0 


2 


0138 


Story Boards 


0 


1 


1 


0 


2 


0139 : 


Non-fiction Books (commercial) 


0 


0 


2 


0 


2 


0149 


Ref Index: Film for TV Program 


0 


0 


2 


0 


2 


0150 : 


Scripts for TV Program 


0 


0 


2 


0 


2 


0152 Video Tapes, completed TV Prog 


0 


0 


2 


0 


2 


0155 


Info/Idea Notebook for Writers 


0 


0 


2 


0 


2 


0157 : 


Sound Track: A-V Presentation 


0 


1 


1 


0 


2 


0159 


Projecu Newsletter 


0 


0 


1 


1 
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Primary Output Category 


Project Focus 


Output 

Totals 

(20) 


Set 

FGHI 

Codine 


Products 


Res 

(5) 


Dev 

(7) 


Dif 

(5) 


Eval 

(3j j 


0175 


Pre-sch Lrng Activities, Parent 


0 


0 


2 


0 


2 


0180 


Pilot Instr Program (movie) 


0 


1 


1 


0 


2 


0192 


Specs: Data Gathering Instrmnt 


0 


1 


1 


0 


2 


0194 


Learner Achievement Data 


1 


0 


1 


0 


2 


0197 


Descr of Research Variables 


1 


0 


1 


0 


2 


0209 


Speech Texts 


0 


0 


2 


0 


2 


0216 


Dissertations 


2 


0 


0 


0 


2 


0219 


Student Performance Objectives 


0 


2 


0 


0 


2 


0231 


List: Acquisition/Data 


0 


1 


1 


0 


2 


0232 


Specs: Output Revisions 


0 


2 


0 


0 


2 


0233 


Curricular Recommendations 


0 


2 


0 


0 


2 


0236 


Single Concept Learning Unit 


0 


2 


0 


0 


2 


0238 


Model of Content Discipline 


2 


0 


0 


0 


2 


0239 


Compendium of Measures 


2 


0 


0 


0 


2 


0240 


Working Papers 


1 


1 


0 


0 


2 


0242 


Data Handling Codes 


0 


0 


0 


2 


2 


0245 


Computer Printouts 


2 


0 


0 


0 


2 


0260 


Summative Evaluation Data 


0 


0 


1 


1 


2 


0269 


Correspondence 


0 


0 


2 


0 


2 


0270 


Computer Program Documentation 


0 


0 


2 


0 


2 


0274 


Usage Report 


0 


0 


2 


0 


2- 


0282 


Memos/Directives/Bulletins 


0 


1 


0 


1 


2 


0290 


Census Data 


0 


0 


0 


2 


2 


0305 


Project Promo Presentation Pkg 


0 


1 


1 

i 


0 


2 


0310 


Script: Film/Tape Presentation 


0 


0 


2 


0 


2 


0320 


Trng Program in Group Dynamics 


2 


0 


0 


0 


2 


0329 


Work Assignment Record 


0 


0 


2 


0 


2 


0003 


Microformed Bibliographies 


1 


0 


0 


0 


1 


0016 


Advisory Committee 


1 


0 


0 


0 


1 


0018 


Enabling Instr Objectives 


0 


1 


0 


0 


1 


0020 


Syndactic Text Instr Materials 


0 


1 


0 


0 


1 


0021 


Remedial Units for Syndactic Texts 


0 


1 


0 


0 


1 


0022 


Single-Concept Instr Films 


0 


] 


0 


0 


1 


0023 


Video Tape Instr Modules 


0 


1 


0 


0 


1 


0025 


Activities Outline: Instr Units 


0 


1 


0 


0 


1 


0027 


Linear Programmed Instr Text 


0 


1 


0 


0 


1 


0028 


Intrinsically Programmed Text 


0 


1 


0 


0 


1 



APPENDIX 1 



(Continued) 





Primary Output Category 




Project 


Focus 


Output 


Set 












Totals 


FGHI 

Coding 


Products 


Res 

( 5 ) 


Dev 

( 7 ) 


Dif 

( 5 ) 


Eval 

( 3 ) 


(20) 


0030 


Materials Presentation Design 


0 


1 


0 


0 


1 


0032 


Norm-Referenced Test Items 


0 


1 


0 


0 


1 


0035 


Parent Instr Unit Use Guide 


0 


0 


1 


0 


1 


0036 


Model: Developing Instr Units 


0 


1 


0 


0 


1 


0037 


Computer-Based Instr Unit 


0 


1 


0 


0 


1 


0039 


Interview Training Procedures 


0 


0 


1 


0 


1 


0041 


Site Visit Checklist 


0 


0 


1 


0 


1 


0043 


Guidelines for Site Visits 


0 


0 


1 


0 


1 


0045 


Site Visitation Schedule 


0 


0 


1 


0 


1 


0050 


Structural Communication Unit 


0 


1 


0 


0 


1 


0060 


Validation Criteria for Data 


0 


0 


1 


0 


1 


0062 


Team Rotation Plan 


0 


0 


1 


0 


1 


0064 


Cost Estimates 


0 


1 


0 


0 


1 


0065 


Test Procedure, I-M Handling System 0 


1 


0 


0 


1 


0066 


List of Project Products 


0 


1 


0 


0 


1 


0068 


Review/Concurrence I-M Handling 


0 


1 


0 


0 


1 


0069 


Design Review Data, I-M System 


0 


1 


0 


0 


1 


0070 


Design for I-M Handling System 


0 


1 


0 


0 


1 


0071 


Design Review Report 


0 


1 


0 


0 


1 


0072 


Specs: Hardware Requirements 


0 


1 


0 


0 


1 


0073 


Specs: Software Requirements 


0 


1 


0 


0 


1 


0075 


Manual for I-M Handling Sys 


0 


1 


0 


0 


1 


0077 


Specs: Automated I-M System 


0 


1 


0 


0 


1 


0079 


Cost Categories: Oper System 


0 


1 


0 


0 


1 


0086 


Bibliographies 


0 


1 


0 


0 


1 


0088 


List of User Needs 


0 


1 


0 


0 


1 


0090 


Report of Operations Analysis 


0 


1 


0 


0 


1 


0093 


Design Concepts, I-M System 


0 


1 


0 


0 


1 


0094 


Automated I-M Handling System 


0 


1 


0 


0 


1 


0106 


Cumulative Post Test 


0 


1 


0 


0 


1 


0107 


Master Tutor Inrstr Unit 


0 


1 


0 


0 


i 


0110 


Content Map for instr Course 


0 


1 


0 


0 


1 


0115 


Project Products Dstrb System 


0 


0 


1 


0 


1 


0125 


Promotional Film 


0 


0 


1 


0 


1 


0128 


Multimedia Leadership Course 


0 


1 


0 


0 


1 






APPENDIX 1 



(Continued) 



Primary Output Category 


Project Focus 


Output 


Set 






Totals 


FGHI 


Products 


Res Dev Dif Eval 


(20) 


Coding 




_(5) ill 112 . m 





0130 Animated Film: Reading Program 

0133 Serialized TV Adventure Script 

0134 Serialized Story Manuscript 

0136 Curriculum for Reading Program 

0137 Ref Manual: Instr in Reading 



0 0 1 
0 0 1 
0 0 1 
0 0 1 
0 0 1 



0 

0 

) 

0 

0 



1 

1 

1 

1 

1 



0141 Film Strip Promotional Unit 

0147 Live-Action Film for TV Show 

0148 Animated Film for TV Show 

0151 Curriculum for Ed TV Program 

0153 Specifications for Facilities 



0 

0 

0 

0 

0 



0 

0 

0 

0 

0 



1 

1 

1 

1 

1 



0 

0 

0 

0 

0 



1 

1 

1 

1 

1 



0161 Diffusion Staff Tmg Program 0 

0162 Demo of Follow-Up to A-V Learning 0 

0163 Training Film for Parents 0 

0167 Conmunity Resource Directory 0 

0168 Demographic Data 0 



0 

0 

0 

0 

0 



1 

1 

1 

1 

1 



0 

0 

0 

0 

0 



1 

1 

1 

1 

1 



0170 Community Resource Data 

0171 Pre-School Guide Activity Data 

0172 Pre-sch Lrng Matter Teacher Use 

0173 Pre-sch Lrng Matter Parent Use 

0174 Pre-sch Lrng Activities, Tchr 



0 0 10 
0 0 10 
0 0 10 
0 0 10 
0 0 10 



1 

1 

1 

1 

1 



0177 Trng Program for Lay Personnel 

0181 Interagency Opns Control Form 

0182 List: Disrrib Agencies/ Points 

0184 Mailing Lists 

0186 Specs: Free Product Dtbg Sites 



0 0 10 
0 0 10 
0 0 10 
0 0 10 
0 0 10 



1 

1 

1 

1 

1 



0189 Data on Product Appeal 

0198 Sight Vocabulary List 

0202 Catalog of Available Products 

0205 Pre-school Text Books 

0206 Educational Greeting Cards 



0 

0 

0 

0 

0 



0 

0 

0 

0 

0 



1 

1 

1 

1 

1 



0 

0 

0 

0 

0 



1 

1 

1 

1 

1 



0208 Plans: Future Project Dvlpmnt 

0212 Educational Calendar 

0213 Children’s Activity Center 

0214 Satellite Management Center 

0215 Learning Kit , commercial 



0 10 0 
0 0 10 
0 0 10 
0 0 10 
0 0 10 



1 

1 

1 

1 

1 



APPENDIX 1 



('Continued) 



Primary Output Categqries 




Project 


Focus 


Out; ut 
Totals 
(20) 


Set 

FGril 

Coding 


Products 


Res 

(5) 


Dev 

(7) 


Di. : 

(5) 


Eva I 
(3) 


0221 


Source List for Objectives 


C 


1 


0 


0 


1 


0225 


Rationale for Data Instrument 


0 


1 

Jm 


0 


0 


1 


0229 


Attitude Measure, Tolerance 


0 


1 


0 


0 


1 


0237 


Model of skills Acquisition 


1 


0 


0 


0 


1 


0251 


School Paper 


0 


0 


1 


0 


1 


0253 


Curriculum Dvlpmnt Guidelines 


0 


0 


1 


0 


1 


0255 


Interview Reports 


0 


1 


0 


0 


1 


0257 


Conference/Workshop Agenda 


0 


1 


0 


0 


1 


0259 


Evaluation Model 


0 


0 


0 


1 


1 


0268 


Microform Copy 


0 


0 


1 


0 


1 


0272 


Publication Proof Copy 


0 


0 


1 


0 


1 


0273 


Publications (not identified) 


0 


0 


1 


0 


1 


0275 


Pro j ect Object ives 


0 


1 


0 


0 


1 


0276 


Articles: House Publications 


0 


1 


0 


0 


1 


0278 


Prospectus/Letters of Intent 


0 


1 


0 


0 


1 


0280 


Specs: Filming Techniques 


0 


1 


0 


0 


1 


0285 


Organizat ional Chart 


0 


0 


0 


1 


1 


0287 


Key Indicators: Condition School 


0 


0 


0 


1 


1 


0288 ' 


Task /Time Allotments 


0 


0 


0 


1 


1 


0291 Management Information Center 


0 


0 


0 


1 


1 


0292 


Mediated Data Display 


0 


0 


0 


1 


1 


0293 


Review/Ref inement Procedures 


0 


0 


0 


1 


1 


0294 Operations Progress Chart 


0 


0 


0 


1 


1 


0297 


Feedback System Classroom Mgmt 


0 


0 


0 


1 


1 


0301 


PPBS Manual 


0 


• 0 


0 


1 


1 


0304 ' 


Computer Simulation: Operation 


0 


1 


0 


0 


1 


0306 


Clearinghouse: RDD&E Outputs 


0 


1 


0 


0 


1 


0307 


Use Guide for Card Sort System 


0 


0 


1 


0 


1 


0308 1 


Data-Handling Cards 


0 


0 


1 


0 


1 


0321 i 


Content Outline Guides/Manuals 


1 


0 


0 


0 


1 


0326 Reporting Procedures 


0 


1 


0 


0 


1 


0327 i 


Coordination/Liaison Procedure 


0 


1 


0 


0 


1 


0334 Theory Papers 


1 


0 


0 


0 


1 


Totals 




83 


282 


295 


99 


759 






265 



APPENDIX 1 



(Conf tnued) 



Primary Output Category 




Project 


Focus 


Output 

Totals 

(20) 


Set 

FGHI 

Coding 


Events 


Res 

(5) 


Dev 

(7) 


Dif 

(5) 


Eval 

(3) 


0241 


Selecting Personnel 


1 


2 


9 


4 


16 


0335 


Revising Existing Materialj 


0 


0 


12 


0 


12 


0176 


Training Workshop/Conference 


3 


1 


3 


2 


9 


0015 


Making Management Decisions 


1 


0 


6 


0 


7 


0114 


Field Tests 


1 


2 


3 


0 


6 


0120 


Acquiring Funds 


0 


0 


6 


0 


6 


0012 


Assigning Work 


2 


1 


1 


1 


5 


0191 


RDD&E on the Job Training 


0 


1 


2 


2 


5 


0246 


Interpreting Data Analysis 


2 


0 


1 


2 


5 


0014 


Disseminating Information 


1 


0 


2 


1 


4 


0119 


Speeches 


l 


0 


2 


1 


4 


0165 


Acquiring Facilities 


0 


1 


3 


0 


4 


02 3 v 


Reviewing Status/ Qual ity 


0 


2 


0 


1 


3 


0264 


Staff Meetings 


1 


0 


1 


1 


3 


0265 


Collecting Data 


0 


0 


0 


3 


3 


0296 


Planning Proj Opns/Guidelines 


0 


0 


1 


2 


3 


0317 


Maintaining Project Progress 


1 


0 


2 


0 


3 


0258 


Development (work) Conference 


0 


2 


0 


0 


2 


0271 


Modifying Systems 


0 


1 


1 


0 


2 


0316 


Coordinating Data Collection 


2 


0 


0 


0 


2 


0318 


Maintaining Fiscal Acctability 


1 


0 


1 


0 


2 


0323 


Making Policy Decisions 


1 


0 


1 


0 


2 


0331 


Expanding Product Availability 


0 


0 


2 


0 


2 


0008 


Evaluating Hardware 


1 


0 


0 


0 


1 


0011 


Acquiring Microform Hardware 


1 


0 


0 


0 


1 


0098 


Info Dissemination Workshop 


0 


1 


0 


0 


1 


0103 


Acquiring Course Materials 


0 


1 


0 


0 


1 


014 3 


Acquiring RV Studio Hardware 


0 


0 


1 


0 


1 


0156 


Pre-use Critique: A-V Program 


0 


0 


1 


0 


1 


0178 


Acquiring Television Sets 


0 


0 


1 


0 


1 


0200 


Policy Setting Seminars 


0 


0 


1 


0 


1 


0243 


Interviewing 


1 


0 


0 


0 


1 


0262 


Selecting of Performance Items 


0 


0 


0 


1 


1 


0277 


Acquiring of Filming Equipment 


0 


1 


0 


0 


1 


0284 


Staff Retreat 


0 


1 


0 


0 


1 



APPENDIX 1 



(Continued) 



Primary Output Categories 


Project Focus 


Output 

Totals 

(20) 


Set 

FGHI 

Coding 


Conditions 


Res 

(5) 


Dev 

(7) 


Dif 

(5) 


Eval 

(3) 


0311 


Surveying by Telephone 


0 


0 


1 


0 


1 


0312 


Surveying Costs 


0 


0 


1 


0 


1 


0313 


Selecting Field Test Sample 


0 


0 


1 


0 


1 


0314 


Coding Data 


0 


0 


1 


0 


1 


0322 


Negotiating Participation/C^op 


1 


0 


0 


0 


1 


0325 


Acquiring Operational Supplies 


0 


1 


0 


0 


1 


0328 


Conducting Experiments 


0 


1 


0 


0 


1 


0333 


Surveying the Literature 


1 


0 


0 


0 


1 


Totals 


23 


19 


67 


21 


130 



Primary Output Categories 




Project 


Focus 


Output 

Totals 

(20) 


Set 

FGHI 

Coding 


Conditions 


Res 

(5) 


Dev 

(7) 


Dif 

(5) 


Eval 

P) 


0010 


Adequate Staff 


1 


1 


4 


1 


7 


0100 


Staff Welfare 


1 


1 


2 


1 


5 


0058 


Quality Output 


1 


1 


2 


0 


4 


0193 


Intr a- Agency Coordination 


1 


0 


3 


0 


4 


0009 


Managed Money Resources 


2 


1 


0 


0 


O 


0057 


De on Making Structure 


0 


1 


1 


1 


3 


0101 


Intra/ Inter Agency Relations 


0 


1 


2 


0 


3 


0109 


Staff Morale 


0 


1 


2 


0 


3 


0247 


Adequate Facilities/Equipment 


2 


0 „ 


1 


0 


3 


0054 


Staff Awareness /Acceptance 


0 


0 


2 


0 


2 


0084 


Interagency Coordination 


0 


0 


1 


1 


2 


0111 


Inter/ Intra Agency Cooperation 


0 


2 


0 


0 


2 


0126 


Relationship with TV Networks 


0 


0 


2 


0 


2 


0127 


Adherence to Project Philosophy 


0 


1 


1 


0 


2 


0179 


Consulting Services 


0 


0 


2 


0 


2 


0196 


Effective Communications 


1 


0 


1 


0 


2 


0207 


Project Management Structure 


0 


1 


1 


0 


2 


0263 


Monitored Project Operations 


1 


0 


1 


0 


2 


0004 


Project Publicity 


1 


0 


0 


0 


1 


0005 


Adequate Field Setting 


1 


0 


0 


0 


1 
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(Continued) 



Primary Output Category 


Project Focus 


Output 


Set 






Totals 


FGHI 


Conditions 


Res Dev Dif Eval 


(20) 


Coding 




(5) (7) (5) (3) 





0056 


Compatible Work/Expense Rates 


0 


0 


1 


0 


1 


0059 


Operation Efficiency, Project 


0 


0 


1 


0 


1 


0112 


Project A antability 


0 


1 


0 


0 


1 


0113 


Site Personnel Welfare 


1 


0 


0 


0 


1 


0116 


Managed Production Personnel 


0 


0 


1 


0 


1 


0124 


Atmosphere: Feeling Effective 


0 


0 


1 


0 


1 


0166 


Facilitated Product Distrib 


0 


0 


1 


0 


1 


0254 


Community /Parent Involvement 


0 


0 


1 


0 


1 


0283 


Adequate Information Flow 


0 


1 


0 


0 


1 


0295 


Healthy Atmosphere: Criticism 


0 


0 


0 


1 


1 


0298 


Maintained Program Operations 


0 


0 


0 


1 


1 


0299 


Test Construction Service 


0 


0 


0 


1 


1 


0300 


Measurement Service 


0 


0 


0 


1 


1 


0302 


Informed Public 


0 


0 


0 


1 


1 


0315 


Pro j ect Leadership 


0 


0 


1 


0 


1 


0319 


Adequate Community Interface 


1 


0 


0 


0 


1 


0330 


Adequate Support Linkages 


0 


0 


1 


0 


1 


0336 


Coordinated/Compatible Outputs 


0 


0 


1 


0 


1 



Totals 


f .* 


13 


14 


36 


10 


73 


Grand Totals 




119 


315 


398 


130 


962 



O 

me 
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APPENDIX 2 



Cluster Categories of Outputs, and the Number of Outputs 
Identified Within Them by Project Focus 





Output Cluster Category 




Project 


Focus 




Output 


Set 

DE 


Products 


Kes 


Dev ] 


Diff 


Eval 


Totals 

(20) 


Coding 




(5) 


(7) 


(5) 


(3) 




01 


Reports/Contracts 


21 


25 


47 


29 


122 


04 


Work Specif icat ions /Procedures 


5 


45 


21 


7 


78 


27 


Data (including printouts) 


10 


26 


25 


14 


75 


02 


Data Instruments /Techniques 


20 


22 


18 


14 


74 


08 


Plans /Designs /Theories /Models 


8 


44 


14 


7 


73 


05 


Work Management Systems 


7 


20 


23 


12 


62 


03 


Guides /Manuals 


3 


18 


29 


6 


56 


10 


Publications 


4 


6 


36 


6 


52 


07 


Resource Lists 


2 


31 


11 


0 


44 


14 


Instruc tnl Units/Use w/Staff 


2 


4 


30 


3 


39 


12 


Instructnl Units/ In-School 


0 


26 


3 


0 


29 


13 


Instructnl Units/Extra-School 


0 


0 


18 


0 


18 


11 


Promotional Materials 


0 


1 


16 


0 


17 


28 


Taxonomies 


0 


5 


0 


0 


5 


29 


Recommendations 


0 


4 


0 


0 


4 


36 


Curricular Objectives 


0 


3 


0 


0 


3 


37 


Critiques 


0 


2 


1 


0 


3 


06 


Data Management Systems 


1 


0 


1 


0 


2 


09 


Equipment /Tools /Facilities 


0 


0 


2 


0 


2 


42 


Prediction Variables 


0 


0 


0 


1 


1 



Totals 


83 282 


295 


99 


759 





Output Cluster Category 


Project Focus 


Output 

Totals 


Set DE 


Events 


Res 


Dev 


Diff 


Eval 


(20) 


Coding 




15L 


m 


(5) 


(3) 




32 


Recruiting/ Placing Staff 


3 


3 


10 


5 


21 


33 


Collecting/Analyzing Data 


4 


2 


5 


5 


16 


52 


Adapting Materials 


0 


0 


12 


0 


12 


43 


Establishing Opns Parameters 


0 


0 


9 ■ 


2 


11 


17 


Workshops/ Institutes 


3 


4 


3 


0 


10 



ri/ , 



APPENDIX 2 



(Continued) 



Output Cluster Category 


Project Focus 


Output 
Tn rals 


Set DE 


Events 


Res 


Dev 


Dlff 


Eval 


1U L C*1 O 
(20) 


Coding 




(5) 


(7) 


(5) 


(3) 




40 


Acquiring Supplies/Facilities 


2 


3 


5 


0 


10 


31 


Field Trials/Experiments 


1 


3 


3 


0 


7 


38 


Training 


0 


1 


2 


4 


7 


51 


Acquiring Funds 


0 


0 


6 


0 


6 


48 


Maintaining Accountability 


2 


0 


3 


0 


5 


18 


Presentations 


1 


0 


2 


1 


4 


19 


Benchmark Decisions 


2 


0 


1 


1 


4 


50 


Disseminating Info/Outputs 


1 


0 


2 


1 


4 


30 


Committee/Consultant Meetings 


0 


1 


1 


1 


3 


15 


Staff Meetings 


1 


0 


0 


1 


2 


16 


Conferences /Seminars 


0 


1 


1 


0 


2 


39 


Updating of Machine Systems 


0 


1 


1 


0 


2 


49 


Establishing Field Sites 


1 


0 


1 


0 


2 


- 47 


Coordinating Efforts 


2 


0 


0 


0 


2 


Totals 




23 


19 


67 


21 


130 



Output Cluster Category 


Project Focus 


Output 

Totals 

(20) 


Set DE 
Coding 


Conditions 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


21 


Working Environment/Atmosphere 


3 


2 


11 


3 


19 


20 


Cooperative Relationships 


2 


5 


6 


1 


14 


34 


Coordinated Efforts 


1 


0 


8 


1 


10 


22 


Quality Assurance 


1 


3 


3 


0 


7 


23 


Accountability 


2 


2 


1 


1 


6 


25 


Resource Accessibility 


1 


0 


2 


1 


4 


41 


Effective Communication 


1 


1 


2 


0 


4 


44 


Provision for Services 


0 


0 


2 


2 


4 


45 


Informed Public 


1 


0 


0 


1 


2 


26 


Available Field Sites 


1 


0 


0 


0 


1 


35 


Acceptance/Adoption of Outputs 


0 


0 


1 


0 


1 


46 


Trained Staff 


0 


1 


0 


0 


1 


Totals 


13 


14 


36 


10 


73 


Grand 


Total.' 


119 


315 


398 


130 


962 



APPENDIX 3 



Primary Categories of Outputs, Organized Within Cluster 
Categories, and the Number of Outputs Identi f led 
Within Them by Project Focus 



Set 






Project Focus 


— 


DE FG11I 


Products 


Res 


Dev Diff Eval 


Total 


Coding 




(5) 


(7) (5) (3) 


(20) 



OUTPUT CLUSTER 



01 




Reports / Contracts 


21 


25 


47 


29 


122 






PRIMARY OUTPUT CATEGORIES 












01 


0158 


Evaluation Report. 


1 


1 


12 


10 


24 


01 


0006 


Terminal Report. 


6 


3 


'4 


3 


16 


01 


0C89 


Status Condition Report. 


1 


2 


5 


6 


14 


01 


0099 


Progress Report. 


4 


5 


3 


1 


13 


01 


0007 


Proposal . 


2 


6 


1 


1 


10 


01 


0040 


Description of Program/Project. 


1 


3 


3 


2 


9 


01 


0144 


Financial Report. 


1 


0 


6 


0 


7 


01 


0230 


Technical Report. 


3 


0 


1 


2 


6 


01 


0053 


Abstracted Descript: Programs. 


0 


0 


4 


0 


4 


01 


0146 


Contracts for Services 


0 


0 


3 


0 


3 


01 


0286 


Survey Reports. 


0 


0 


2 


1 


3 


01 


0261 


Formative/Field Test Eval Data. 


0 


0 


0 


3 


3 


01 


0216 


Dissertations . 


2 


0 


0 


0 


2 


01 


0274 


Usage Report. 


0 


0 


2 


0 


2 


01 


0068 


Review Procedures, I-M Handling. 


0 


1 


0 


0 


1 


01 


0071 


Review of Design Report. 


0 


1 


0 


0 


1 


01 


0090 


Report of Operations Analysis. 


0 


1 


0 


0 


1 


01 


0255 


Interview Reports. 


0 


1 


0 


0 


1 


01 


0278 


Prospectus/Letters of Intent. 


0 


1 


0 


0 


1 


01 


0047 


A/V Descriptions of Programs. 


0 


0 


1 


0 


1 



OUTPUT CLUSTER 



04 


Work Specif i cations /Procedures 


5 45 


21 


7 


78 







APPENDIX 3 



(Continued) 



Set 






Project Focus 




DE FGHI 
Coding 


Products 


Res 

(5) 


Dev Diff Eval 
(7) (5) (3) 


Total 

(20) 



PRIMARY OUTPUT CATEGORIES 



04 


0095 


Description/Specs for Output. 


04 


0220 


Data Analysis Procedures. 


04 


0017 


Terminal Instr Objectives. 


04 


0222 


Data Collection Procedures. 


04 


0154 


Specifications for Personnel. 


04 


0197 


Descr of Research Variables. 


04 


0074 


Specs: Operation Requirements, 


04 


0076 


Specs: I-M Handling Sub-Systems. 


04 


0091 


Performance Specs: I-M Systems. 


04 


0092 


Design Alternates for Evaluation. 


04 


0117 


Program (within project) Plan, 


04 


0121 


Script, Film. 


04 


0192 


Specs: Data Gathering Instrmnt. 


04 


0232 • 


Specs: Output Revisions. 


04 


0042 


Selection Criteria; Sites etc. 


04 


0055 


Report Specifications . 


04 


0150 


Scripts for TV Program. 


04 


0209 


Speech Texts. 


04 


0310 


Script: Film/Tape Presentation. 


04 


0321 


Content Outline Guides/Manuals. 


04 


0018 


Enabling Instr Objectives. 


04 


0019 


Content Outline, Instructional. 


04 


0065 


Test Procedure, I-M Handling System. 


04 


0066 


List of Project Products. 


04 


0067 


Implementing Plan, I-M Handling. 


04 


0072 


Specs: Hardware Requirements. 


04 


0073 


Specs: Software Requirements. 


04 


0077 


Specs: Automated I-M System. 


04 


0078 


Major Task Descriptions. 


04 


0079 


Cost Categories: Oper System. 


04 


00811 


Interview Guide. 


04 


0085 


Project Operational Plan. 


04 


0275 


Project Objectives. 


04 


0280 


Specs: Filming Techniques. 


04 


0029 


Site Assessment Criteria. 



0 

2 

0 

1 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



7 

3 

5 

3 

0 

0 

2 

2 

2 

2 

1 

1 

1 

2 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0 



0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

1 

1 

0 

1 

1 

2 

2 

2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 



0 

0 

0 

0 

2 

0 

0 

0 

0 

0 

1 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

? 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

3 

1 

1 

1 

1 

1 

1 
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bJ bn bn 



APPENDIX 3 



(Continued) 



Set 

DE FGHI 
Cc ding 


Products 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


Dif f 
(5) 


Eval 

(3) 


04 


0060 


Validation Criteria for Data. 


0 


0 


1 


0 


1 


04 


0118 


Formulated Policy. 


0 


0 


1 


0 


1 


04 


03 33 


Serialized TV Adventure Script. 


0 


0 


1 


0 


1 


04 


0136 


Curriculum for Reading Program. 


0 


0 


1 


0 


1 


04 


0151 


Curriculum for Ed TV Program. 


0 


0 


1 


0 


1 


04 


0153 


Specifications for Facilities. 


0 


0 


1 


0 


1 


04 


0186 


Specs: Free Product Dtbg Sites. 


0 


0 


1 


0 


] 


04 


0198 


Sight Vocabulary List. 


0 


0 


1 


0 


1 


04 


0013 


Project Time Line/Schedule. 


0 


0 


0 


1 


1 


04 


0293 


Review/Refinement Procedures . 


0 


0 


0 


1 


1 



OUTPUT CLUSTER 



27 




Data (including printouts) 


10 


26 


25 


14 


75 






PRIMARY OUTPUT CATEGORIES 












27 


0044 


Interview Data/Information. 


0 


1 


1 


3 


5 


27 


0169 


Diffusion Impact Data. 


0 


3 


2 


0 


5 


27 


0188 


Normative Data. 


0 


0 


O 

£ 


3 


5 


27 


0235 


Baseline Data. 


3 


2 


0 


0 


5 


27 


0001 


Prototype Tryout Data. 


1 


3 


0 


0 


4 


27 


0185 


Needs Identification Data. 


0 


2 


1 


1 


4 


27 


0199 


Item Analysis Data. 


0 


2 


2 


0 


4 


27 


0038 


Research Data. 


1 


1 


1 


0 


3 


27 


0190 


Product Review/Critique. 


0 


3 


0 


0 


3 


27 


0195 


Social Interaction Data. 


1 


0 


1 


1 


3 


27 


0244 


Coded Data. 


1 


0 


2 


0 


3 


27 


0267 


Indexes . 


0 


0 


3 


0 


3 


27 


0033 


Learner Data File. 


0 


1 


1 


0 


2 


27 


0087 


List of Potential Output Users. 


0 


2 


0 


0 


2 


27 


0194 


Learner Achievement Data 


1 


0 


1 


0 


2 


27 


0245 


Computer Printouts 


2 


0 


0 


0 


2 


27 


0260 


Summative Evaluation Data. 


0 


0 


1 


1 


2 


27 


0261 


Formative/Field Test Eval Data. 


0 


2 


0 


0 


2 


27 


0289 


Directories . 


0 


0 


0 


2 


2 * 


27 


0290 


Census Data. 


0 


0 


0 


2 


2 
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APPENDIX 3 



(Continued) 



Set 

DE FGHI 

Coding 


Products 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


D7 f f 
(5) 


Eval 

(3) 


27 


0047 


A/V Descriptions of Programs. 


0 


0 


1 


0 


1 


27 


0064 


Cost Estimates. 


0 


1 


0 


0 


1 


27 


0069 


Design Review Data, I-M system 


0 


1 


0 


0 


1 


27 


0088 


List of User Needs. 


0 


1 


0 


0 


1 


27 


0095 


Description/Specs for Output. 


0 


1 


0 


0 


1 


27 


0168 


Demographic Data. 


0 


0 


1 


0 


1 


27 


0170 


Community Resource Data. 


0 


0 


1 


0 


1 


27 


0171 


Pre-school Guide Activity Data. 


0 


0 


1 


0 


1 


27 


0189 


Data on Product Appeal. 


0 


0 


1 


l 


1 


27 


023.1 


Handling Sys: Project Outputs. 


0 


0 


1 


0 


1 


27 


0231 


List : Acquis it ion/ Data. 


0 


0 


1 


0 


1 


27 


0286 


Survey Reports. 


0 


0 


0 


1 


1 


OUTPUT CLUSTER 



02 




Data Instruments/Techniques 


20 


22 


18 


14 


74 






PRIMARY OUTPUT CATEGORIES 












02 


0187 


Descriptive Data Tests, Subject. 


14 


1 


3 


1 


19 


02 


0082 


Questionnaire . 


2 


5 


6 


2 


15 


02 


0061 


Observation System. 


1 


1 


2 


5 


9 


02 


0217 


Observation Record Form. 


0 


4 


2 


0 


6 


02 


0224 


Attitude Meas, Cognitive Value. 


0 


3 


0 


2 


5 


02 


0031 


Criterion-Based Test Item Pool. 


0 


3 


0 


0 


3 


02 


0081 


Interview Guide. 


0 


3 


0 


0 


3 


02 


0309 


Worksheets: Proj Documentation. 


0 


0 


3 


0 


3 


02 


0239 


Compendium of Measures. 


2 


0 


0 


0 


2 


02 


( '49 


Progress Check Test. 


0 


1 


0 


1 


2 


02 


0242 


Data Handling Codes? 


0 


0 


0 


2 


2 


02 


0220 


Data Analysis Procedures. 


1 


0 


0 


0 


1 


02 


0106 


Cumulative Post Test. 


0 


1 


0 


0 


1 


02 


0181 


Interagency Opns Control Form. 


0 


0 


1 


0 


1 


02 


0308 


Data-Handling Cards. 


0 


0 


1 


0 


1 


02 


0096 


Evaluation Checklist. 


0 


0 


0 


1 


1 
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APPENDIX 3 



(Continued) 



Set 

DE r^Hi 
Codir & 


r 

Products 

j 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


OUTPUT CLUSTER 


08 




Plans/Designs /Theories/ Models 


£ 


44 


14 


7 


73 






PRIMARY OUTPUT CATEGORIES 












08 


0218 


Quality Control Schema. 


C 


12 


0 


0 


12 


08 


0256 


Field Test Plan. 


0 


5 


4 


0 


9 


08 


0281 


Evaluation Plan/Design. 


0 


4 


1 


1 


6 


08 


0002 


Research Design. 


3 


1 


0 


1 


5 


08 


0117 


Program (within project) Plan. 


0 


0 


2 


3 


5 


08 


0226 


Rationale for Objvs and Msmt. 


0 


4 


0 


0 


4 


08 


0067 


Implementing Plan, I-M Handling. 


0 


2 


1 


'o 


3 


08 


0085 


Project Operational Plan. 


0 


3 


0 


0 


3 


08 


0279 


Conceptual Framework. 


0 


2 


1 


0 


3 


08 


0238 


Model of Content Discipline. 


2 


0 


0 


0 


2 


08 


0240 


Working Papers. 


1 


1 


0 


0 


2 


08 


0053 


Abstracted Descript: Programs. 


0 


2 


0 


0 


2 


08 


0118 


Formulated PoJ icy . 


0 


0 


2 


0 


2 


08 


0222 


Data Collection Procedures. 


0 


0 


2 


0 


2 


08 


0237 


Model ox Skills Acquisition. 


1 


0 


0 


0 


1 


08 


0334 


Theory Papers. 


1 


0 


0 


0 


1 


08 


0019 


Content Outline, Instructional. 


0 


1 


0 


0 


1 


08 


0030 


Materials Presentation Design. 


0 


1 


0 


0 


1 


08 


0036 


Model: Developing Instr Units. 


0 


1 


0 


0 


1 


08 


0070 


Design for I-M Handling System. 


0 


1 


0 


0 


1 


08 


0093 


Design Concepts, I-M System. 


0 


1 


0 


0 


1 


08 


0208 


Plans: Future Project Dvlpmnt. 


0 


1 


0 


0 


1 


08 


0225 


Rationale for Data Instrument. 


0 


1 


0 


0 


1 


08 


0257 


Conference/Workshop Agenda. 


0 


1 


0 


0 


1 


08 


0062 


Team Rotation Plan. 


0 


0 


1 


0 


1 


08 


0220 


Data Analysis Procedures. 


0 


0 


0 


1 


1 


08 


0259 


Evaluation Model. 


0 


0 


0 


1 


1 



APPENDIX 3 



(Continued) 



Uet | 




Project Focus 




DE FGHIi 


Products 


Res 


Dev Diff Eval 


Total 


Coding 1 






(7) (5) (3) 


(20) 



OUTPUT CLUSTER 



05 




Work Management Systems 


7 


20 


23 


12 


62 






PkIHARY OUTPUT CATEGORIES 












05 


003 3 


Project lime Line/Schedule, 


4 


6 


7 


2 


19 


05 


0324 


Budget Allocation:]. 


1 


3 


2 


1 


7 


05 


0i45 


Intraagency Opns Control Form. 


1 


0 


3 


0 


4 


05 


0083 


PERf Chart. 


0 


j 


0 


0 


3 


05 


0211 


Handling Sys: Project Outputs. 


0 


0 


3 


0 


3 


05 


0218 


Quality Control Schema. 


0 


2 


0 


0 


2 


r ' 
J 


0282 


Memos /Directives/Bullet ins. 


0 


1 


0 


1 


2 


05 


0269 


Correspondence . 


0 


0 


2 


0 


2 


05 


0329 


Work Assignment Record. 


0 


0 


2 


0 


2 


05 


0033 


Learner Data File. 


0 


0 


0 


2 


2 


05 


0016 


Advisory Committee. 


1 


0 


G 


0 


1 


05 


0067 


Implementing Plan, I-M Handling. 


0 


1 


0 


0 


1 


05 


0094 


Automated I-M Handling System. 


0 


1 


0 


0 


1 


05 


0281 


Evaluation Plan/Design. 


0 


1 


0 


0 


1 


05 


0326 


Reporting Procedures. 


0 


1 


0 


0 


1 


05 


0327 


Coordination/Liaison Procedure. 


0 


1 


0 


0 


1 


05 


0040 


Description of Program/Project. 


0 


0 


1 


0 


1 


05 


0045 


Site Visitation Schedule. 


0 


0 


1 


0 


1 


05 


0115 


Project Products Dstrb System. 


0 


0 


1 


0 


1 


05 


0184 


Mailing Lists. 


0 


0 


1 


0 


1 


05 


0285 


Organizational Chart. 


0 


0 


0 


1 


1 


05 


0288 


Task/Time Allotments. 


0 


0 


0 


1 


1 


05 


0291 


Management Information Center. 


0 


0 


0 


1 


1 


05 


0292 


Mediated Data Display. 


0 


0 


0 


1 


1 


05 


0294 


Operations Progress Chart. 


0 


0 


0 


1 


1 


05 


0297 


Feedback System Classroom Mgmt. 


0 


0 


0 


1 


1 
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APPENDIX 3 



( Continued) 



Set 






Project Focuj 




UE FGHI 
Coding 


Products 


Res 

(5) 


Dev Diff Eval 
(7) (5) a) 


Total 

(20) 



OUTPUT CLUSTER 




03 Guide? /Manuals 3 18 29 6 56 



PRIMARY OUTPUT FGOPIES 



03 


0164 


Diffusion Manual/Guidelines. 


0 


0 


7 


0 


7 


03 


0034 


Instructional Unit Use Guide. 


1 


4 


1 


0 


6 


03 


0108 


Supplementary Instr Materials. 


0 


4 


0 


0 


4 


03 


0055 


Report Specifications . 


0 


0 


2 


2 


4 


03 


0081 


Interview Guide. 


2 


0 


0 


1 


3 


03 


0095 


Description/Specs for Output. 


0 


1 


2 


0 


3 


03 


0138 


Story Boards. 


0 


1 


1 


0 


2 


03 


0249 


Staff Development Trng Unit. 


0 


2 


0 


0 


2 


03 


0149 


Ref Index: Film for TV Program. 


0 


0 


2 


0 


2 


03 


0155 


Info/Id^a Notebook for Writers. 


0 


0 


2 


0 


2 


03 


0075 


Manual for I-M Handling System. 


0 


1 


0 


0 


1 


03 


0078 


Major Task Descriptions. 


0 


1 


0 


0 


1 


03 


0096 


Evaluation Checklist. 


0 


1 


0 


0 


1 


03 


0110 


Content Map for Instr. Course. 


0 


1 


0 


0 


1 


03 


0217 


Observation Record Form. 


0 


1 


0 


0 


1 


03 


0281 


Evaluation Plan/Pesign. 


0 


1 


0 


0 


1 


03 


0002 


Research Design. 


0 


0 


1 


0 


1 


03 


0035 


Parent Instr Unit Use Guide. 


0 


0 


1 


0 


1 


03 


0041 


Site Visit Checklist. 


0 


0 


1 


0 


1 


03 


0042 


Selection Criteria: Sites etc. 


0 


0 


1 


0 


1 


03 


0043 


Guidelines for Site Visits. 


0 


c 


1 


0 


1 


03 


0134 


Serialized Story Manuscript. 


0 


0 


1 


0 


1 


03 


0137 


Ref Manual: Instr in Reading. 


0 


0 


1 


0 


1 


03 


0175 


Pre-sch T rng Activities, Parent. 


0 


0 


1 


0 


1 


03 


0253 


Curriculum Dvlpmnt Guidelines. 


0 


0 


1 


0 


1 


03 


0267 


Indexes . 


0 


0 


1 


0 


1 


03 


0270 


Computer Program Documentation. 


0 


0 


1 


0 


1 


03 


0307 


Use Guide for Card Sort System. 


0 


0 


1 


0 


1 


03 


0029 


Site Assessment Criteria. 


0 


0 


0 


1 


1 


03 


0222 


Data Collection Procedures. 


0 


0 


0 


1 


i 


03 


0301 


PPBS Manual. 


0 


0 


0 


1 


1 



f\t., t 
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APPENDIX 3 



(Continued) 



Set 




Project Focus 




DE FGHI 


Products 


Res 


Dev Diff Eval 


Total 


CodinR 




J*L 


_1Z) (-5) (3) 


(20) 



OUTPUT CLUSTER 



10 




Publications 


4 


6 


36 


6 


52 






PRIMARY OUTPUT CATEGORIES 












10 


0266 


Journal. 


0 


0 


7 


0 


7 


10 


0201 


Professional Report /Monograph. 


2 


2 


1 


1 


6 


10 


0203 


Learning Activity Books. 


0 


0 


6 


0 


6 


10 


0097 


Journal Article, Published. 


2 


2 


1 


0 


5 


10 


0135 


Television Program Magazine. 


0 


0 


5 


0 


5 


10 


0332 


Magazine/Newspaper Articles. 


0 


0 


3 


0 


3 


10 


0040 


Description of Program/Project. 


0 


0 


2 


0 


2 


10 


0139 


Non-fiction Books (comnercial) . 


0 


0 


2 


0 


2 


10 


0159 


Project Newsletter. 


0 


0 


1 


1 


2 


10 


0267 


Indexes. 


0 


0 


2 


0 


2 


10 


0252 


Information Brochures. 


0 


0 


0 


2 


2 


10 


0286 


Survey Reports. 


0 


0 


0 


2 


2 


10 


0227 


Compendium: Instr Objvs/Meas. 


0 


1 


0 


0 


1 


10 


0276 


Articles: House Publications. 


0 


1 


0 


0 


1 


10 


0053 


Abstracted Descript: Programs. 


0 


0 


1 


0 


1 


10 


0183 


List of Resource Personnel. 


0 


0 


1 


0 


1 


10 


0251 


School Paper. 


0 


0 


1 


0 


1 


10 


0268 


Mfcroform Copy. 


0 


0 


1 


0 


1 


10 


0272 


Publication Proof Copy. 


0 


0 


1 


0 


1 


10 


0273 


Publications (not identified). 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


07 




Resource Lists 


2 


31 


11 


0 


44 



PRIMARY OUTPUT CATEGORIES 



07 U227 Compendium: Instr Objvs/Meas 



0 13 



0 



0 



13 



APPENDIX 3 



(Continued) 



Set Project Focus 

DE FGHI Products Res Dev Diff Eval Total 

Coding (5) (7) (5) (3) (20 ) 



07 


0046 


List of Field Settings. 


i 


4 


4 


0 


9 


07 


0228 


Compendium: lnstr Objectives 


0 


4 


0 


0 


u 


07 


0040 


Description of Program/Project. 


0 


0 


2 


0 


2 


07 


0183 


List of Resource Personnel. 


0 


1 


1 


0 


2 


07 


0003 


Microformed Bibliographies. 


1 


0 


0 


0 


1 


07 


0031 


Criterion-Based Test Item Pool. 


0 


1 


0 


0 


1 


07 


0032 


Norm-Referenced Test Items. 


0 


1 


0 


0 


1 


07 


0086 


Bibliographies . 


0 


1 


0 


0 


1 


07 


0095 


Description/Specs for Output. 


0 


1 


0 


0 


1 


07 


0221 


Source List for Objectives 


0 


1 


0 


0 


1 


07 


0224 


Attitude Meas, Cognitive Value 


0 


1 


0 


0 


1 


07 


0229 


Attitude Measure, Tolerance 


0 


1 


0 


0 


1 


07 


0231 


List : Acquis it ion/ Data 


0 


1 


0 


0 


1 


07 


0306 


Clearinghouse: RDD&E Outputs. 


0 


1 


0 


0 


1 


07 


0167 


Community Resource Directory. 


0 


0 


1 


0 


1 


07 


0182 


List: Distrib Agencies/Points. 


0 


0 


1 


0 


1 


07 


0202 


Catalog of Available Products. 


0 


0 


1 


0 


1 


07 


0289 


Directories . 


0 


0 


1 


0 


1 



OUTPUT CLUSTER 



14 lnstructnl Units/Use w/Staff 2 4 30 3 39 



PRIMARY OUTPUT CATEGORIES 



14 


0249 


Staff Development Trng Unit, 


0 


0 


12 


0 


12 


14 


0250 


Pupil Svc Personnel Trng Unit. 


0 


0 


10 


0 


10 


14 


0160 


Training Film for Educators. 


0 


0 


3 


2 


5 


14 


0303 


Computer lnstr in Computer Use. 


0 


3 


0 


0 


3 


14 


0320 


Trng Program in Group Dynamics. 


2 


0 


0 


0 


2 


14 


0304 


Computer Simulation: Operation. 


0 


1 


0 


0 


1 


14 


0039 


Interview Training Procedures. 


0 


0 


1 


0 


1 


14 


0161 


Diffusion Staff Trng Program. 


0 


0 


1 


0 


1 


14 


0162 


Demo of Follow-Up to A-V Learning. 


0 


0 


1 


0 


1 


14 


0163 


Training Film for Parents. 


0 


0 


1 


0 


1 


14 


0177 


Trng Program for Lay Personnel. 


0 


0 


1 


0 


1 


14 


0024 


Audio Tape lnstr Modules. 


0 


0 


0 


1 


1 



A 
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APPENDLX 3 



(Continued) 



Set 




Project Focus 




DE 


FGH1 


Products 


Res 


Dev 


Diff 


Eval 


Total 


Coding 




(5) 


(7) 


_J5) 


(3) 


(20) 


OUTPUT CLUSTER 


12 




Instructnl Units/In-School 


0 


26 


3 


0 


29 






PRIMARY OUTPUT CATEGORIES 












12 


0026 


Computer Asst/Ext Instr Units. 


0 


2 


0 


0 


2 


12 


0104 


Instructional Package, Medium. 


0 


2 


0 


0 


2 


12 


0105 


Instructional Package* Small. 


0 


2 


0 


0 


2 


12 


0108 


Supplementary Instr Materials. 


0 


2 


0 


0 


2 


12 


0236 


Single Concept Learning Unit. 


0 


2 


0 


0 


2 


1? 


0020 


Syndactic Text Instr Materials. 


0 


1 


0 


0 


1 


12 


0021 


Remedial Units for Syndactic Texts. 


0 


1 


0 


0 


1 


12 


0022 


Single-Concept Instr Films. 


0 


1 


0 


0 


1 


12 


0023 


Video Tape Instr Modules. 


0 


1 


0 


0 


1 


12 


0024 


Audio Tape Instr Mocules. 


0 


1 


0 


0 


1 


12 


0025 


Activities Outline: Instr Units. 


0 


1 


0 


0 


1 


12 


0027 


Linear Programmed Instr Text. 


0 


1 


0 


0 


1 


12 


0028 


Intrinsically Programed Text. 


0 


1 


0 


0 


1 


12 


0037 


Computer-Based Instr Unit. 


0 


1 


0 


0 


1 


12 


0050 


Structural Communication Unit. 


0 


1 


0 


0 


1 


12 


0052 


Pane) Book Visuals: Audio Tape. 


0 


1 


0 


0 


1 


12 


0107 


Master Tutor Instr Unit. 


0 


1 


0 


0 


1 


12 


0128 


Multimedia Leadership Course. 


0 


1 


0 


0 


1 


12 


0157 


Sound Track: A-V Presentation. 


0 


1 


0 


0 


1 


12 


0180 


Pilot Instr Program (movie). 


0 


1 


0 


0 


1 


12 


0303 


Computer Instr in Computer Use. 


0 


1 


0 


0 


1 


J? 


0172 


Pre-sch Lrng Matter Teacher Use. 


0 


0 


1 


0 


1 


12 


0174 


Pre-sch Lrng Activities, Tchr. 


0 


0 


1 


0 


1 


12 


0205 


Pre-school Text Books. 


0 


0 


1 


0 


L 


OUTPUT CLUSTER 


13 




Instructnl Units/Extra-School 


0 


0 


18 


0 


18 



*-• 



O 
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APPENDIX 3 



(Continued) 



Set 


‘1 


Project Focus 




DE 


FGHI 


Products 


Res 


Dev 


Diff 


Eval 


Total 


Coding 




(5) 


V) 


(5) 


O) 


(20) 






PRIMARY OUTPUT CATEGORIES 












13 


0129 


Television Program. 


0 


0 


2 


0 


2 


13 


0131 


Live Action Film: Reading Prog. 


0 


0 


2 


0 


2 


13 


0132 


Video Tape, TV Prog Component. 


0 


0 


O 


0 


2 


13 


0152 


Video Tapes, completed TV Program. 


0 


0 


2 


0 


2 


13 


0024 


Audio Tape Instr Modules. 


0 


0 


1 


0 


l 


13 


0052 


Panel Book Visuals: Audio Tape, 


0 


0 


1 


0 


1 


13 


0130 


Animated Film: Reading Program. 


0 


0 


1 


0 


1 


13 


0147 


Live-Action Film for TV Show, 


0 


0 


1 


0 


1 


13 


0148 


Animated Film for TV Show. 


0 


0 


1 


0 


1 


1? 


0157 


Sound Track: A-V Presentation. 


0 


0 


1 


0 


1 


13 


0173 


Pre-sch Lrng Matter Parent Use. 


0 


0 


1 


0 


1 


13 


0175 


Pre-sch Lrng Activities, Parent. 


0 


0 


1 


0 


1 


13 


0180 


Pilot Instr Program (movie). 


0 


0 


1 


0 


1 


13 


0215 


Learning Kit, commercial. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


11 




Promotional Materials 


0 


1 


16 


0 


17 






PRIMARY OUTPUT CATEGORIES 












11 


014C 


Promotional Phonograph Records. 


0 


0 


5 


0 


5 


11 


0204 


Learning Activity Toys/Games. 


0 


0 


3 


0 


3 


11 


0252 


Information Brochures. 


0 


0 


3 


0 


3 


11 


0305 


Project Promo Presentation Pkg. 


0 


1 


1 


0 


2 


11 


0125 


Promotional Film. 


0 


0 


1 


0 


1 


11 


0141 


Film Strip Promotional Unit. 


0 


0 


1 


0 


1 


11 


0206 


Educational Greeting Cards. 


0 


0 


1 


0 


1 


11 


0212 


Educational Calendar. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


28 




Taxonomies 


0 


5 


0 


0 


5 



APPENDIX 3 



(Continued) 



Set 

UE FGHI 
Cod ins 


Products 


Project Focus 


Total 

(20) 


Res 


Dev 

L2L- 


Diff 

(5) 


Eval 

_UL 






PRIMARY OUTPUT CATEGORIES 












28 


0223 


Taxonomy for Learner Outcomes. 


0 


5 


0 


0 


5 


OUTPUT CLUSTER 


29 




Recomm^ndat ions 


0 


A 


0 


0 


A 






PRIMARY OUTPUT CATEGORIES 












29 


0095 


Description/Specs for Output. 


0 


2 


0 


0 


2 


29 


0233 


Curriculum Recommendations* 


0 


2 


0 


0 


2 



OUTPUT CLUSTER 



36 




Curricular Objectives 


0 


3 


0 


0 


3 






PRIMARY OUTPUT CATEGORIES 












36 


0219 


Student Performance Objectives. 


0 


2 


0 


0 


2 


36 


0228 


Compendium: Instr Objectives. 


0 


1 


0 


0 


1 


OUTPUT CLUSTER 



37 Critiques 0210 3 



PRIMARY OUTPUT CATEGORIES 












0190 Product Review/Critique. 


0 


1 


1 


0 


2 


0201 Professional Report /Monograph. 


0 


1 


0 


0 


1 



O 
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nf 
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APPENDIX 3 



(Continued) 



Set 






Project Focus 




DE FGHI 
Coding 


Products 


Res 

(5) 


Dev Diff Evai 
(7) (5) (3) 


Total 

(20) 



OUTPUT CLUSTER 



06 




Data Management Systems 


1 


0 


1 


0 


2 






PRIMARY OUTPUT CATEGORIES 












06 


0220 


Data Analysis Procedures. 


1 


0 


0 


0 


1 


06 


0270 


Computer Program Documentation. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


09 




Equipment/Tools/Facilit ies 


0 


0 


2 


0 


2 






PRIMARY OUTPUT CATEGORIES 












09 


0213 


Children's Activity Center. 


0 


0 


1 


0 


1 


09 


0214 


Satellite Management Center. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


42 




Prediction Variables 


0 


0 


0 


1 


1 






PRIMARY OUTPUT CATEGORIES 












42 


0287 


Key Indicators: Condition School. 


0 


0 


0 


1 


1 



O 

ERLC 
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APPENDIX 3 



(Continued) 



Set 






Project Focus ! 




DE FGHI 


Events 


Res 


Dev Diff Eval 


Total 


Coding 




(5) 


(7) (5) (3) ! 


(20) 



OUTPUT CLUSTER 



32 




Recruiting/Placing Staff 


3 


3 


10 


5 


21 






PRIMARY OUTPUT CATEGORIES 












32 


0241 


Selecting Personnel. 


1 


2 


9 


4 


16 


32 


0012 


Assigning Work. 


2 


1 


1 


1 


5 



OUTPUT CLUSTER 



33 




Collecting/Analyzing Data. 


4 


2 


5 


5 


16 






PRIMARY OUTPUT CATEGORIES 












33 


0246 


Interpreting Data Analysis. 


2 


0 


1 


2 


5 


33 


0265 


Collecting Data. 


0 


0 


0 


3 


3 


33 


0243 


Interviewing. 


1 


0 


0 


0 


1 


33 


0333 


Surveying the Literature. 


1 


0 


0 


0 


1 


33 


0103 


Acquiring Course Materials. 


0 


1 


0 


0 


1 


33 


0234 


Reviewing Status/Quality. 


0 


1 


0 


0 


1 


33 


0156 


Pre-use Critique: A-V Program. 


0 


0 


1 


0 


1 


33 


0311 


Surveying by Telephone. 


0 


0 


1 


0 


1 


33 


0312 


Surveying Costs. 


0 


0 


1 


0 


1 


33 


0314 


Coding Data. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


52 




Adapting Materials 


0 


0 


12 


0 


12 



PRIMARY OUTPUT CATEGORIES 












0335 Revising Existing Materials. 


0 


0 


12 


0 


12 



ERLC 
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APPENDIX 3 



(Continued) 



Set 






Project Focus 




DE FGHI 
Coding 


Events 


Res 

All. 


Dev Dlff Eval 

AH ill HA 


Total 

(20) 



OUTPUT CLUSTER 



43 




Establishing Opns Parameters 


0 


0 


9 


2 


11 






PRIMARY OUTPUT CATEGORIES 












43 


0015 


Making Management Decisions. 


0 


0 


5 


0 


5 


43 


0296 


Planning Proj Opns/Gulde lines . 


0 


0 


1 


2 


3 


43 


0331 


Expanding Product Availability. 


0 


0 


2 


0 


2 


43 


0323 


Making Policy Decisions. 


0 


0 


1 


0 


1 



OUTPUT CLUSTER 



17 


Workshops/ Ins ti tutes 


3 


4 


3 


0 


10 


PRIMARY OUTPUT CATEGORIES 



17 


0176 


Training Workshop/Conference. 


3 


0 


3 


0 


6 


17 


0258 


Development (work) Conference. 


0 


2 


0 


0 


2 


17 


0098 


Info Dissemination Workshop. 


0 


1 


0 


0 


1 


17 


0284 


Staff Retreat. 


0 


1 


0 


0 


1 



OUTPUT CLUSTER 



40 




Acquiring Supplles/Facllltles 


2 


3 


5 


0 


10 






PRIMARY OUTPUT CATEGORIES 












40 


0165 


Acquiring Facilities. 


0 


1 


3 


0 


4 


40 


0008 


Evaluating Hardware. 


1 


0 


0 


0 


1 


40 


0011 


Acquiring Microform Hardware. 


1 


0 


0 


0 


1 


40 


0277 


Acquiring of Filming Equipment. 


0 


1 


0 


0 


1 


40 


0325 


Acquiring Operational Supplies. 


0 


1 


0 


0 


1 



APPENDIX 3 



(Continued) 



Set 

DE FGH I 
Coding 


Events 


Project Focua 


To t a 1 

_i30 1 


Res 

U* 1 


Dev 

XT)- 


Diff 

JJ± 


Eval 

- 13 ) 


40 


0143 


Acquiring TV Studio Hardware. 


0 


0 


1 


0 


1 


40 


0178 


Acquiring Television Sets. 


0 


0 


1 


0 


l 


OUTPUT CLUSTER 


31 




Field Trials/Experiments 


1 


3 


3 


0 


7 






PRIMARY OUTPUT CATEGORIES 












31 


0114 


Field Tests. 


1 


2 


3 


0 


6 


31 


0328 


Conducting Experiments. 


0 


1 


0 


0 


1 


OUTPUT CLUSTER 


38 




Training 


0 


1 


2 


4 


7 






PRIMARY OUTPUT CATEGORIES 












38 


0191 


RDD&E on the Job Training. 


0 


1 


0 


2 


5 


38 


0176 


Training Workshop/Conference. 


0 


0 


0 


2 


2 



OUTPUT CLUSTER 



51 


Acquiring Funds 


0 


0 


6 


0 


6 




PRIMARY OUTPUT CATEGORIES 












51 


0120 Acquiring Funds. 


0 


0 


6 


0 


6 
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APPENDIX 3 



(Continued) 



Set 






Project Focua 




DE FGH1 
Coding 


Events 


Rea 

AIL 


Dev Uiff Eval 

All LA L2A 


To t ml 
(20) 



OUTTVr CLUSTER 



48 




Maintaining Accountability 


2 


0 


3 


0 


5 






PRIMARY OUTPUT CATEGORIES 












48 


0317 


Maintaining Project Progress. 


1 


0 


2 


0 


3 


48 


0318 


Maintaining Flacal Acctability. 


1 


0 


1 


0 


2 


OUTPUT CLUSTER 


18 




Presentations 


1 


0 


2 


1 


4 






PRIMARY OUTPUT CATEGORIES 












18 


0119 


Speeches. 


1 


0 


2 


1 


4 






OUTPUT CLUSTER 












50 




Disseminating Info/Outputs 


1 


0 


2 


1 


4 






PRIMARY OUTPUT CATEGORIES 












50 


0014 


Disseminating Information. 


1 


0 


2 


1 


h 


OUTPUT CLUSTER 


19 




Benchmark Decisions 


2 


0 


1 


1 


4 
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APPENDIX 3 



(Continued) 



Set 






Project Focus 




DE FGHI 
Cod inn 


Events 


Res 

(5) 


Dev Dlff Eval 
(7) (5) (3) 


Tot Ai 
120) 



PRIMARY OUTPUT CATEGORIES 



19 


0015 


Making Management Decisions. 


1 


0 


1 


0 


2 


19 


0323 


Making Policy Decisions. 


1 


0 


0 


0 


1 


19 


0262 


Selecting of Performance Items. 


0 


0 


0 


1 


1 


OUTPUT CLUSTER 


30 




Committee/Consultant Meetings 


0 


1 


1 


1 


3 






PRIMARY OUTPUT CATEGORIES 












30 


0234 


Reviewing StatuB/Quallty . 


0 


1 


0 


1 


2 


30 


0264 


Staff Meetings. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


15 




Staff Meetings 


1 


0 


0 


1 


2 






PRIMARY OUTPUT CATEGORIES 












15 


0264 


Staff Meetings. 


1 


0 


0 


1 


2 


OUTPUT CLUSTER 


16 


Conferences/Seminars 


0 


1 


1 


0 


2 






PRIMARY OUTPUT CATEGORIES 












16 


0176 


Training Workshop/Conference. 


0 


1 


0 


0 


1 


16 


0200 


Policy Setting Seminars. 


0 


0 


1 


0 


1 
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APPENDIX 3 
(Continued) 



Set 






Project Focus 


— 


DE FCHI 


Events 


Res 


Dev Diff Eval 


Total 


Coding 




(5) 


(7) (5) (3) 


(20) 



OUTPUT CLUSTER 



39 




Updating of Machine Systems 


0 


1 


1 


0 


2 






PRIMARY OUTPUT CATEGORIES 












39 


0271 


Modifying Systems. 


0 


1 


1 


0 


2 


OUTPUT CLUSTER 


47 




Coordinating Efforts 


2 


0 


0 


0 


2 






PRIMARY OUTPUT CATEGORIES 












47 


0316 


Coordinating Data Collection. 


2 


0 


0 


0 


2 


OUTPUT CLUSTER 


49 




Establishing Field Sites 


1 


0 


1 


0 


2 






PRIMARY OUTPUT CATEGORIES 












49 


0322 


Negotiating Participation/Coop. 


1 


0 


0 


0 


1 


49 


0313 


Selecting Field Test Sample. 


0 


0 


1 


0 


1 



O 

ERIC 



A 



APPENDIX 3 



(Continued) 



Set 


1 “ 




Project Focus 




DE FGHI 
Coding 


Conditions 


Res 

(5) 


l>ev Diff Eval ' 


Total 

(20) 



OUTPUT CLUSTER 



21 




Working Environment/ Atmosphere 


3 


2 


11 


3 


19 






PR DIARY OUTPUT CATEGORIES 












21 


0010 


Adequate Staff. 


1 


0 


4 


1 


6 


21 


0100 


Staff Welfare. 


1 


1 


2 


1 


5 


21 


0109 


Staff Morale. 


0 


1 


2 


0 


3 


21 


0247 


Adequate Facilities/Equipment. 


1 


0 


0 


0 


1 


21 


0054 


Staff Awareness/ Acceptance. 


0 


0 


1 


0 


1 


21 


0057 


Decision Making Structure. 


0 


0 


1 


0 


1 


21 


0124 


Atmosphere: Feeling Effective. 


0 


0 


1 


0 


L 


21 


0295 


Healthy Atmosphere: Criticism. 


0 


0 


0 


1 


1 


OUTPUT CLUSTER 


20 




Cooperative Relationships 


2 


5 


6 


1 


14 






PRIMARY OUTPUT CATEGORIES 












20 


0101 


Intra/Inter Agency Relations. 


0 


1 


2 


0 


3 


20 


0057 


Decision Making Structure. 


0 


1 


0 


1 


2 


20 


0111 


Inter/Intra Agency Cooperation. 


0 


2 


0 


0 


2 


20 


0207 


Project Management Structure. 


0 


1 


1 


0 


2 


20 


0126 


Relationship with TV Networks. 


0 


0 


2 


0 


2 


20 


0113 


Site Personnel Welfare. 


1 


0 


0 


0 


1 


20 


0319 


Adequate Community Interface. 


1 


0 


0 


0 


1 


20 


0254 


Community/Parent Involvement. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


34 




Coordinated Efforts 


1 


0 


8 


1 


10 



APPENDIX 3 



(Continued) 



Set 






Project Focus 




DE FGHI 
Coding 


Conditions 


Res 


Dev Diff Eval 
(7) (5) (3) 


Total 

(20) 



PRIMARY OUTPUT CATEGORIES 



34 


0193 


Intra-agency Coordination. 


1 


0 


3 


0 


4 


34 


0084 


Inter-agency Coordination. 


0 


0 


1 


1 


2 


34 


0059 


Operation j Efficiency, Project. 


0 


0 


1 


0 


1 


34 


0116 


Managed Production Personnel. 


0 


0 


1 


0 


1 


34 


0315 


Project Leadership. 


0 


0 


1 


0 


1 


34 


0336 


Coordinated /Compatible Outputs. 


0 


0 


1 


0 


1 



OUTPUT CLUSTER 



22 




Quality Assurance 


1 


3 


3 


0 


7 






PRIMARY OUTPUT CATEGORIES 












22 


0058 


Quality Output. 


1 


1 


2 


0 


4 


22 


0127 


Adherence to Project Philosophy. 


0 


1 


1 


0 


2 


22 


0263 


Monitored Project Operations. 


0 


1 


0 


0 


1 



OUTPUT CLUSTER 



23 




Accountability 


2 


2 


1 


1 


6 






PRIMARY OUTPUT CATEGORIES 












23 


0009 


Managed Money Resources. 


2 


1 


0 


0 


3 


23 


0112 


Project Accountability. 


0 


1 


0 


0 


1 


23 


0056 


Compatible Work/Expense Rates. 


0 


0 


1 


0 


1 


23 


0263 


Monitored Project Operations. 


0 


0 


0 


1 


1 




301 
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APPENDIX 3 



(Continued) 



Set 




Project Focus 




DE FGHI 


Conditions 


Res 


Dev 


Diff 


Eval 


Total 


Coding 




(5) 


(7) 


(5) 


(3) 


(20) 


OUTPUT CLUSTER 


25 


Resource Accessibility 


1 


0 


2 


1 


4 


PRIMARY OUTPUT CATEGORIES 


25 0247 


Adequate Facilities/Equipment . 


1 


0 


1 


0 


2 


25 0330 


Adequate Support Linkages. 


0 


0 


1 


0 


1 


25 0298 


Maintained Program Operations. 


0 


0 


0 


1 


1 



OUTPUT CLUSTER 



41 




Effective Communication 


1 


1 


2 


0 


4 






PRIMARY OUTPUT CATEGORIES 












41 


0196 


Effective Communications. 


1 


0 


1 


0 


2 


41 


028j 


Adequate Information Flow. 


0 


1 


0 


0 


1 


41 


0166 


Facilitated Product Dlstrlb. 


0 


0 


1 


0 


1 



OUTPUT CLUSTER 



44 




Provision for Services 


0 


0 


2 


2 


4 






PRIMARY OUTPUT CATEGORIES 












44 


0179 


Consulting Services. 


0 


0 


2 


0 


2 


44 


0299 


Test Construction Service. 


0 


0 


0 


1 


1 


44 


0300 


Measurement Service. 


0 


0 


0 


1 


.1 
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APPENDIX 3 



(Continued) 



Set 




Project Focus 




DE 


FGHI 


Conditions 


Res 


Dev 


Diff 


Eval 


Total 


Cod in^ 




(5) 


(7) 


(5) 


(3) 


(20) 


OUTPUT CLUSTER 


45 




Informed Public 


1 


0 


0 


1 


2 






PRIMARY OUTPUT CATEGORIES 












45 


0004 


Project Publicity, 


1 


0 


0 


0 


1 


45 


0302 


Informed Public. 


C 


0 


0 


1 


1 


OUTPUT CLUSTER 


26 




Available Field Sites 


1 


0 


0 


0 


1 






PRIMARY OUTPUT CATEGORIES 












26 


0005 


Adequate Field Setting. 


1 


0 


0 


0 


1 


OUTPUT CLUSTER 


35 




Acceptance/Adoption of Outputs. 


0 


0 


1 


0 


1 






PRIMARY OUTPUT CATEGORIES 












35 


0054 


Staff Awareness/Acceptance. 


0 


0 


1 


0 


1 


OUTPUT CLUSTER 


46 




Trained Staff 


0 


1 


0 


0 


1 



me 
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APPENDIX 3 



(Continued) 



Set 




Project Focus 




DE FGHI 


Conditions 


Res 


Dev 


Diff 


Eval 


Total 


Coding 




(5) 


(7) 


(5) 


(3) 


(20) 


PRIMARY OUTPUT CATEGORIES 




46 0010 Adequate Staff. 


0 


1 


0 


0 


1 



APPENDIX 4 



Primary Categories of Outputs, Organized Within Cluster 
Categories, and the Number of Outputs Analyzed 
Within Them by Project Focus 



Set 

DE FGHI 
Coding 


Products 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(Z1 


Diff 

(5) 


Eval 

(3) 


CLUSTER CATEGORY 


01 




Reports /Contracts 


6 


6 


9 


5 


26 






PRIMARY CATEGORIES 












01 


0099 


Progress Report. 


1 


3 


1 


0 


5 


01 


0158 


Evaluation Report. 


0 


0 


2 


3 


5 


01 


0040 


Description of Program/Project. 


1 


1 


2 


0 


4 


01 


0007 


Propc 'al . 


1 


2 


0 


0 


3 


01 


0144 


Financial Report. 


1 


0 


2 


0 


3 


01 


0006 


Terminal Report. 


2 


0 


0 


0 


2 


01 


0053 


Abstract Progras Descriptions 


0 


0 


2 


0 


2 


01 


0089 


Status Condition Report. 


0 


0 


0 


1 


1 


01 


0230 


Technical Report. 


0 


0 


0 


1 


1 


CLUSTER CATEGORY 


08 




Plans/Designs/Theories/Models 


4 


17 


2 


3 


26 






PRIMARY CATEGORIES 












08 


0002 


Research Design. 


3 


1 


0 


1 


5 


08 


0218 


Quality Control Schema. 


0 


5 


0 


0 


5 


08 


0226 


Rationale For Objvs and Msmt . 


0 


2 


0 


0 


2 


08 


0281 


Evaluation Plan/Design. 


0 


1 


0 


1 


2 


08 


0039 


Materials Presentation Design. 


0 


1 


0 


0 


1 


08 


0036 


Model: Developing Instr Units. 


0 


1 


0 


0 


1 


08 


0067 


Implementing Plan, I-M Handling. 


0 


1 


0 


0 


1 


08 


0085 


Project Operational Plan. 


0 


1 


0 


0 


1 


08 


0117 


Program (w/ in project) Plan. 


0 


0 


1 


0 


1 


08 


0208 


Plans: Future Project Dvmt. 


0 


1 


0 


0 


1 



O 

ERLC 



sc: 



APPENDIX 4 



(Continued) 



Set 

DE FGHI 
Coding, 


Products 


Project Focus 


Total 

m 


Res 

V) 


Dev 

JJ1-. 


Diff 

ill_ 


Eval 

(3) 


08 


0225 


Rationale for Data Instrument. 


0 


1 


0 


0 


1 


08 


0240 


Working Papers. 


0 


1 


0 


0 


1 


08 


0019 


Content Outline, Instructional 


0 


1 


0 


0 


l 


08 


0256 


Field Test Plan. 


0 


0 


1 


0 


1 


08 


0259 


Evaluation Model. 


0 


0 


0 


1 


1 


08 


0334 


Theory Papers. 


1 


0 


0 


0 


1 


CLUSTER CATEGORY 



04 




Work Specifications/Procedures 


2 


13 


5 


1 


21 






PRIMARY CATEGORIES 












04 


0095 


Description/Specs for Output. 


0 


3 


0 


0 


3 


04 


0220 


Data Analysis Procedures. 


1 


2 


0 


0 


3 


04 


0017 


Terminal Instr Objectives. 


0 


2 


0 


0 


2 


04 


0209 


Speech Texts. 


0 


0 


2 


0 


2 


04 


0222 


Data Collection Procedures. 


1 


1 


0 


0 


2 


04 


0018 


Quality Control Schema. 


0 


1 


0 


0 


1 


04 


0019 


Student Performance Objectives. 


0 


1 


0 


0 


1 


04 


0042 


Selection Criteria: Sites etc. 


0 


0 


0 


1 


1 


04 


0055 


Report Specifications. 


0 


0 


1 


0 


1 


04 


0076 


Specs: I-M Handling Sub-systems. 


0 


1 


0 


0 


1 


04 


0091 


Performance Specs: I-M System. 


0 


1 


0 


0 


1 


04 


0121 


Script, Film. 


0 


0 


1 


0 


1 


04 


0136 


Curriculum for Reading Program. 


0 


0 


1 


0 


1 


04 


0192 


Specs: Data Gathering Instrmnt. 


0 


1 


0 


0 


1 



CLUSTER CATEGORY 



02 


Data Instruments/Techniques 


6 7 


2 


5 


20 
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APPENDIX 4 



(Continued) 



Set 

DE FGHI 
Coding 


Products 


| 


Project Focus | 


Total 

(20) 


Res 
1 (5) 


Dev 

(7) 


Dif f 
(5) 


Eval ! 
(3) I 






PRIMARY CATEGORIES 












02 


0082 


Questionnaire . 


1 


2 


1 


1 


5 


02 


0187 


Descriptive Data Tests, Subject. 


2 


0 


1 


1 


4 


02 


0061 


Observation bystem. 


1 


0 


0 


2 


3 


02 


0224 


Attitude Meas, Cognitive Value. 


0 


2 


0 


0 


2 


02 


0031 


Criterion-Based Test Item Pool. 


0 


1 


0 


0 


1 


02 


0049 


Progress Check Test. 


0 


1 


0 


0 


1 


02 


0081 


Interview Guide. 


0 


1 


0 


0 


1 


02 


0220 


Data Analysis Procedures. 


1 


0 


0 


0 


1 


02 


0239 


Compendium of Measures. 


1 


0 


0 


0 


1 


02 


0242 


Data Handling Codes. 


0 


0 


0 


1 


1 



CLUSTER CATEGORY 



05 


Work Management Systems 


4 


5 


7 


2 


18 


PRIMARY CATEGORIES 



05 


0013 


Project Time Line/Schedule. 


3 


0 


1 


0 


4 


05 


0324 


Budget Allocations. 


0 


2 


1 


0 


3 


05 


0016 


Advisory Committee. 


1 


0 


0 


0 


1 


05 . 


0033 


Learner Data File. 


0 


0 


0 


1 


1 


05 


0040 


Description of Program/Project. 


0 


0 


1 


0 


1 


05 


0083 


PERT Chart. 


0 


1 


0 


0 


1 


05 


0094 


Automated I-M Handling System. 


0 


1 


0 


0 


1 


05 


0115 


Project Products Dstrb System. 


0 


0 


1 


0 


1 


05 


0211 


Handling Sys: Project Outputs. 


0 


0 


1 


0 


1 


05 


0218 


Quality Control Schema. 


0 


1 


0 


0 


1 


05 


0269 


Correspondence . 


0 


0 


1 


0 


1 


05 


0297 


Feedback System Classroom Mgmt. 


0 


0 


0 


1 


1 


05 


0329 


Work Assignment Record* 


0 


0 


1 


0 


1 
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APPENDIX 4 



(Continued) 



"" 1 r — 


Set ' 




Project Focus 




i)E FGHI j Products 


1 Res 


Dev 


Diff 


Eval 


Total 


Coding 


i (5) 


(7) 


(5) 


. . (1) 


(20) 


CLUSTER CATEGORY 


07 Resource Lists 


1 


8 


4 


0 


13 







PRIMARY CATEGORIES 












07 


0040 


Description of Program/Project. 


0 


0 


2 


0 


2 


07 


0227 


Compendium: Instr Objvs/Meas. 


0 


2 


0 


0 


2 


07 


0228 


Compendium: Instr Objectives. 


0 


2 


0 


0 


2 


07 


0003 


Micro formed Bibliographies . 


1 


0 


0 


0 


1 


07 


0046 


List of Field Settings. 


0 


0 


1 


0 


1 


07 


0086 


Bibliographies . 


0 


1 


0 


0 


1 


07 


0183 


List of Resource Personnel. 


0 


1 


0 


0 


1 


07 


0221 


Source List for Objectives. 


0 


1 


0 


0 


1 


07 


0289 


Directories . 


0 


0 


1 


0 


1 


07 


0306 


Clearinghouse: RDD&E Outputs. 


0 


1 


0 


0 


1 



CLUSTER CATEGORY 


12 




Instructnl Units/ In-School 


0 


13 


0 


0 


13 






PRIMARY CATEGORIES 












12 


0020 


Syndactic Text Instr Materials. 


0 


1 


0 


0 


1 


12 


0022 


Single-Concept Instr Films. 


0 


1 


0 


0 


1 


12 


0023 


Video Tape Instr Modules. 


0 


1 


0 


0 


1 


12 


0024 


Audio Tape Instr Modules. 


0 


1 


0 


0 


1 


12 


0026 


Computer Asst/Ext Instr Units. 


0 


1 


0 


0 


1 


12 


002 7 


Linear Programmed Instr Text. 


0 


1 


0 


0 


1 


12 


0028 


Intrinsically Porgrammed Text. 


0 


1 


0 


0 


1 


12 


0037 


Computer-Based Instr Unit. 


0 


1 


0 


0 


1 


12 


0050 


Structural Communication Unit. 


0 


1 


0 


0 


1 


12 


0104 


Instructional Package, Medium. 


0 


1 


0 


0 


1 


12 


0107 


Master Tutor Instr Unit. 


0 


1 


0 


0 


1 


12 


0236 


Single Concept Learning Unit. 


0 


1 


0 


0 


1 


12 


0303 


Computer Instr in Computer Use. 


0 


1 


0 


0 


1 



0 
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APPENDIX 4 



(Continued) 



Set 




i 


Project Focus 1 




DE FGHI 


Products 


j Res 


Dev Diff Eval j 


Total 


Coding 




1 (5) 


(7) (5) (3) 1 


(20) 



CLUSTER CATEGORY 



27 


Data (including printouts) 


2 8 


1 


2 


13 


PRIMARY CATEGORIES 



27 


0001 


Prototype Tryout Data. 


i 


1 


0 


0 


2 


27 


0044 


Interview Data/ Information . 


0 


1 


0 


1 


2 


27 


0199 


Item Analysis Data. 


0 


2 


0 


0 


2 


27 


0033 


Learner Data File. 


0 


1 


0 


0 


1 


27 


0038 


Research Data. 


0 


0 


1 


0 


1 


27 


0064 


Cost Estimates. 


0 


1 


0 


0 


1 


27 


0169 


Design Review Data, I-M System. 


0 


1 


0 


0 


1 


27 


0190 


Product Review/Critique . 


0 


1 


0 


0 


1 


27 


0235 


Baseline Data. 


1 


0 


0 


0 


1 


27 


0260 


Summative Evaluation Data. 


0 


0 


0 


1 


1 


CLUSTER CATEGORY 


03 




Guides/Manuals 


0 


4 


3 


2 


9 







PRIMARY CATEGORIES 












03 


0029 


Site Assessment Criteria. 


0 


0 


0 


1 


1 


03 


0034 


Instructional Unit Use Guide. 


0 


1 


0 


0 


1 


03 


0055 


Report Specifications . 


0 


0 


1 


0 


1 


03 


0081 


Interview Guide. 


0 


0 


0 


1 


1 


03 


0110 


Content Map for Instr Course. 


0 


1 


0 


0 


1 


03 


0138 


Story Boards. 


0 


1 


0 


0 


1 


03 


0155 


Info/Idea Notebook for Writers. 


0 


0 


1 


0 


1 


03 


0249 


Staff Development Trng Unit. 


0 


1 


0 


0 


1 


03 


0267 


Indexes . 


0 


0 


1 


0 


1 



APPENDIX 4 



(Continued) 



- — . — . - ■ “ — ' ■ 


Set 

DE FGHI 
Coding 


Products 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


CLUSTER CATEGORY 


10 




Publications 


0 


2 


5 


0 


7 






PRIMARY CATEGORIES 












10 


0332 


Magazine/Newspaper Articles, 


0 


0 


3 


0 


3 


10 


0053 


Abstracted Descript: Programs. 


0 


0 


1 


0 


1 


10 


0097 


Journal Article, Published. 


0 


1 


0 


0 


1 


10 


0266 


Journal. 


0 


0 


1 


0 


1 


10 


0227 


Compendium: Instr Objvs/Meas. 


0 


1 


0 


0 


1 


CLUSTER CATEGORY 


14 




Instructnl Units/Use w/Staff 


0 


3 


2 


0 


5 






PRIMARY CATEGORIES 












14 


0303 


Computer Instr in Computer Use. 


0 


2 


0 


0 


2 


14 


0304 


Computer Simulation: Operation. 


0 


1 


0 


0 


1 


14 


0039 


Interview Training Procedures. 


0 


0 


1 


0 


1 


14 


0249 


Staff Development Trng Unit. 


0 


0 


1 


0 


1 


CLUSTER CATEGORY 


13 




Instructnl Uni ts/Extra-School 


0 


0 


3 


0 


3 






PRIMARY CATEGORIES 












13 


0129 


Television Program. 


0 


0 


1 


0 


1 


13 


0147 


Live-Action Film for TV Show. 


0 


0 


1 


0 


1 


13 


0157 


Sound Track: A-V Presentation. 


0 


0 


1 


0 


1 



O ~~ ’ 
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300 



APPENDIX 4 



(Continued) 



Set 

DE FGHI 
Coding 


Products 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 






CLUSTER CATEGORY 














06 




Data Management Systems 




1 


0 


1 


0 


2 






PRIMARY CATEGORIES 














06 


0220 


Data Analysis Procedures. 




1 


0 


0 


0 


1 


06 


0270 


Computer Program Documentation. 




0 


0 


1 


0 


1 


CLUSTER CATEGORY 


28 




Taxonomies 




0 


2 


0 


0 


2 






PRIMARY CATEGORIES 














28 


0223 


Taxonomy for Learner Outcomes. 




0 


2 


0 


0 


2 


CLUSTER CATEGORY 


11 




Promotional Materials 




0 


0 


1 


0 


1 






PRIMARY CATEGORIES 














11 


0125 


Promotional Film. 




0 


0 


1 


0 


1 


CLUSTER CATEGORY 


37 




Critiques 




0 


1 


0 


0 


1 






PRIMARY CATEGORIES 














37 


0201 


Professional Report /Monograph. 




0 


1 


0 


0 


1 
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APPENDIX 4 



(Continued) 



-■ ..... — r 1 ..... 


Se t j 




Project Focus 




DE FGH1 j 
Coding . 


Products 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


Total 

(20) 


CLUSTER CATEGORY 


42 


Prediction Variables 


0 


0 


0 


1 


1 


PRIMARY CATEGORIES 


42 0287 


Key Indicators: Condition School. 


0 


0 


0 


1 


1 







O 



/> 
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APPENDIX 4 



(Continued) 





Set 






Project 


Focus 




DE 


FGHI 


Events 


Res 


Dev Diff 


Eval 


Total 


Coding 




1(5) 


(7) 


(5) 


(3) 


! (20) 


CLUSTER CATEGORY 


32 




Recruit ing/ Placing Staff 


2 


3 


8 


1 


14 






PRIMARY CATEGORIES 












32 


0241 


Selecting Personnel. 


1 


2 


7 


1 


11 


32 


0012 


Assigning Work. 


1 


1 


1 


0 


3 


CLUSTER CATEGORY 


43 




Establishing Opns Parameters 


0 


0 


9 


2 


11 






PRIMARY CATEGORIES 












43 


0015 


Making Management Decisions. 


0 


0 


5 


0 


5 


43 


0296 


Planning Proj Opns/Guidelines . 


0 


0 


1 


2 


3 


43 


0331 


Expanding Product Availability. 


0 


0 


2 


0 


2 


43 


0323 


Making Policy Decisions. 


0 


C 


1 


0 


1 



CLUSTER CATEGORY 



51 


Acquiring Fuads 


0 


0 


5 


0 


5 


PRIMARY CATEGORIES 


51 0120 


Acquiring Funds. 


0 


0 


5 


0 


5 


CLUSTER CATEGORY 


33 


Collecting/Analyzing Data 


1 


c 


1 


2 


4 



O 

ERIC 
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APPENDIX 4 



(Contin"ed) 



Set 

DE FGHI 
Coding 


Events 

_ 


Project Focus 


Total 

(20) 


Res 

(5) 


DiV 

o> 


Diff 

(5) 


Eval 

(3) 






PRIMARY CATEGORIES 












33 


0243 


Interviewing. ' 


1 


0 


0 


0 


1 


33 


0156 


Pre-use Critique: A-V Program. 


0 


0 


1 


0 


1 


33 


0246 


Interpreting Data Analysis. 


0 


0 


0 


1 


1 


33 


0265 


Collecting Data. 


0 


0 


0 


1 


1 


CLUSTER CATEGORY 


48 




Maintaining Accountability 


2 


0 


2 


0 


4 






PRIMARY CATEGORIES 












48 


0317 


Maintaining Project Progress. 


1 


0 


1 


0 


2 


48 


0318 


Maintaining Fiscal Acctability. 


1 


0 


1 


0 


2 


CLUSTER CATEGORY 


50 




Disseminating Info /Outputs 


1 


0 


2 


1 


4 






PRIMARY CATEGORIES 












50 


0014 


Tissemir ating Information . 


1 


0 


2 


1 


4 


CLUSTER CATEGORY 


52 




Adapting Materials 


0 


0 


4 


0 


4 






PRIMARY CATEGORIES 












52 


0335 


Revising Existing Materials. 


0 


0 


4 


0 


4 
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APPENDIX A 



(Continued) 



Set 




Project Focus 




DE FGH1 


Events 


Res 


Dev Diff Eval 


Total 


Coding 




(5) 


(7) (5) (3) 


(20) 



CLUSTER CATEGORY 



17 




Workshops/ Inst it utes 


i 


2 


0 


0 


3 






PRIMARY CATEGORIES 












17 


0176 


Training Workshop/Conference. 


i 


0 


0 


0 


1 


17 


0258 


Development (work) Conference. 


0 


1 


0 


0 


1 


17 


028A 


Staff Retreat. 


0 


1 


0 


0 


1 


CLUSTER CATEGORY 


38 




Training 


0 


1 


1 


1 


3 






PRIMARY CATEGORIES 












38 


0191 


RDD&E on the Job Training. 


0 


1 


1 


1 


3 


CLUSTER CATEGORY 


AO 




Acquiring Supplies /Facilities 


2 


0 


1 


0 


3 






PRIMARY CATEGORIES 












AO 


0008 


Evaluating Hardware. 


1 


0 


0 


0 


1 


AO 


0011 


Acquiring Microform Hardware. 


1 


0 


0 


0 


1 


AO 


0165 


Acquiring Facilities. 


0 


0 


1 


0 


1 


CLUSTER CATEGORY 


19 




Benchmark Decisions 


1 


0 


1 


0 


2 



3: 
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APPENDIX 4 



(Continued) 





Set 




Project Focus 




DE FGHI 


Events 


Res 


Dev 


Diff 


Eval 


Total 


Coding 




(5) 


(7) 


HI 


(3) 


(20) 


PRIMARY CATEGORIES 


19 0015 


Making Management Decisions. 


1 


0 


1 


0 


2 



CLUSTER CATEGORY 



31 


Field Trials/Experiments 


i 


i 


0 


0 


2 


PRIMARY CATEGORIES 


31 0114 


Field Tests. 


i 


i 


0 


0 


2 



CLUSTER CATEGORY 



39 




Updating of Machine Systems 


0 


1 


1 


0 


2 






PRIMARY CATEGORIES 












39 


0271 


Modifying Systems . 


0 


1 


1 


0 


2 


CLUSTER CATEGORY 


16 




Conferences/ Seminars 


0 


1 


0 


0 


1 






PRIMARY CATEGORIES 












16 


0176 


Training Workshop/Conference. 


0 


1 


0 


0 


1 



o 
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APPENDIX 4 



(Continued) 



Set 






Project Focus 




DE FGHI 
Coding 


Events 


Res 

(5) 


Dev Diff Eval 
(7) (5) (3) 


Total 

(20) 



r CLUSTER CATEGORY 



18 




Presentations 


0 


0 


.1 


0 


1 






PRIMARY CATEGORIES 












18 


0119 


Speeches . 


0 


0 


1 


0 


1 


CLUSTER CATEGORY 


30 




Committee/Consultant Mettings 


0 


0 


1 


0 


1 






PRIMARY CATEGORIES 












30 


0264 


Staff Meetings. 


0 


0 


1 


0 


1 


CLUSTER CATEGORY 


47 




Coordinat ing Efforts 


1 


0 


0 


0 


1 






PRIMARY CATEGORIES 












47 


0316 


Coordinating Data Collection. 


1 


0 


0 


0 


1 




3:7 
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APPENDIX 4 



(Continued) 



n- - - ■ ■■■•■ 


Set 




Project Focus 




DL FGHI 


Conditions 


Res Dev Diff Eval 


Total 


Coding 




(5) (7) (5) (3) 


(20) 



CLUSTER CATEGORY 



21 




Working Environment /Atmosphere 


i 


2 


9 


2 


14 






PRIMARY CATEGORY 












21 


0010 


Adequate Staff. 


i 


0 


3 


1 


5 


21 


0100 


Staff Welfare. 


0 


1 


2 


0 


3 


21 


0109 


Staff Morale. 


0 


1 


2 


0 


3 


21 


0054 


Staff Awareness/ Acceptance . 


0 


0 


1 


0 


1 


21 


0124 


Atmosphere: Feeling Effective. 


0 


0 


1 


0 


1 


21 


0795 


Healthy Atmosphere: Criticism. 


0 


0 


0 


1 


1 



CLUSTER CATEGORY 



20 


Cooperative Relationships 


i 


5 


6 


0 


12 


PRIMARY CATEGORIES 



20 


0101 


Intra/Inter Agency Relations. 


0 


1 


2 


0 


3 


20 


0111 


Inter/Intra Agency Cooperation. 


0 


2 


0 


J 


2 


20 


0207 


Project Management Structure. 


0 


1 


1 


0 


2 


20 


0126 


Relationship with TV Networks. 


0 


0 


2 


0 


2 


20 


0113 


Site Personnel Welfare. 


1 


0 


0 


0 


1 


20 


0057 


Decision Making Structure. 


0 


1 


0 


0 


1 


20 


0254 


Community/Parent Involvement . 


0 


0 


1 


0 


1 



CLUSTER CATEGORY 



22 


Quality Assurance 


0 


2 


3 


0 


5 
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APPENDIX 4 



(Continued) 



Set 






Project Focus 




DE FGHI 
Coding 


Condi tions 

i 


Res 

(5) 


Dev Diff Eval 
(7) (5) (3) 


Total 

(20) 



PRIMARY CATEGORIES 



22 


0058 


Quality Output. 


0 


1 


2 


0 


3 


22 


0263 


Monitored Project Operations. 


0 


1 


0 


0 


1 


22 


0127 


Adherence to Project Philosophy. 


0 


0 


1 


0 


1 



CLUSTER CATEGORY 



23 




Accountability 


2 


2 


1 


0 


5 






PRIMARY CATEGORIES 












23 


0009 


Managed Money Resources. 


2 


1 


0 


0 


3 


23 


0112 


Project Accountability. 


0 


1 


0 


0 


1 


23 


0056 


Compatible Work/Expense Rates. 


0 


0 


1 


0 


1 


CLUSTEP CATEGORY 


34 




Coordinated Efforts 


0 


0 


4 


1 


5 






PRIMARY CATEGORIES 













34 


0059 


Operations Efficiency, Project. 


0 


0 


1 


0 


1 


34 


0116 


Managed Production Personnel. 


0 


0 


1 


0 


1 


34 


0315 


Project Leadership. 


0 


0 


1 


0 


1 


34 


0336 


Coordinated/Coiupat ible Outputs. 


0 


0 


1 


0 


1 


34 


0084 


Interagency Coordination. 


0 


0 


0 


1 


1 



CLUSTER CATEGORY 



44 Provision for Services 0012 3 



3 3 ® 
o 

ERIC 



APPENDIX 4 



(Continued) 



Set 

DE FGHI 
Coding 


Conditions 


Project Focus 


Total 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 






PRIMARY CATEGORIES 












44 


0179 


Consulting Services . 


0 


0 


1 


0 


1 


44 


0299 


Test Construction Service. 


0 


0 


0 


1 


1 


44 


0300 


Measurement Service . 


0 


0 


0 


1 


1 



CLUSTER CATEGORY 



25 




Resource Accessibility 


0 


0 


0 


1 


1 






PRIMARY CATEGORIES 












25 


0298 


Maintained Program Operations. 


0 


0 


0 


1 


1 






CLUSTER CATEGORY 












26 




Available Field Sites 


1 


0 


0 


0 


1 






PRIMARY CATEGORIES 












26 


0005 


Adequate Field Setting. 


1 


0 


0 


0 


1 


CLUSTER CATEGORY 


35 




Acceptance/Adoption of Outputs 


0 


0 


1 


0 


1 






PRIMARY CATEGORIES 












35 


0054 


Staff Awareness/Acceptance. 


0 


0 


1 


0 


1 
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APPENDIX 4 



(Continued) 



Set 

DE FGHI 
Coding i 




i 


Project Focus 


Total 

(20) 


Conditions 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


CLUSTER CATEGORY 


41 


Effect ive Communication 


0 


1 


0 


0 


1 


PRIMARY CATEGORIES 


41 0283 


Adequate Information Flow. 


0 


1 


0 


0 


1 



CLUSTER CATEGORY 



45 




Informed Public 


i 


0 


0 


0 


1 






PRIMARY CATEGORIES 












45 


0004 


Proj ect Publicity. 


i 


0 


0 


0 


1 


CLUSTER CATEGORY 


46 




Trained Staff 


0 


1 


0 


0 


1 






PRIMARY CATEGORIES 












46 


0010 


Adequate Staff. 


0 


1 


0 


0 


1 
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APPENDIX 5 



Primary Categories of Standards, and the Number of Standards 
Statements Coded Within Them by Project Focus 





Primary Standard Category 




Project Focus 


Totals 

(20) 


Set 

LM Output Standards 

Coding 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


12 


Goal attainment 


18 


35 


31 


18 


102 


13 


Acceptance by others (in proj) 


10 


38 


14 


15 


77 


01 


Completeness of content 


10 


29 


18 


3 


60 


06 


Acceptance by users 


0 


19 


35 


3 


57 


05 


Utility or value 


1 


28 


19 


7 


55 


07 


Personal satisfaction/ feeling 


8 


28 


9 


2 


47 


04 


Communication and clarity 


4 


27 


8 


5 


44 


09 


Lack of errors/discrepancies 


3 


23 


7 


7 


40 


22 


Functions as planned 


6 


14 


8 


7 


35 


11 


Appropriate design /content 


1 


.12 


10 


10 


33 


16 


Compares favorab.1 y 


4 


16 


2 


6 


28 


14 


Acceptance by sponsor 


2 


13 


7 


2 


24 


08 


Agreement/concurrence w/others 


2 


12 


4 


3 


21 


18 


Satisfactory appearance 


5 


7 


3 


1 


16 


19 


Logical criteria 


3 


9 


2 


1 


15 


21 


Sources of variance controlled 


3 




2 


7 


15 


17 


Internally consistent 


2 


.10 


2 


0 


14 


15 


Compliance w/sponser guideline 


1 


3 


9 


0 


13 


30 


Lack of negative feedback 


1 


6 


3 


1 


11 


23 


Successfully constrains/guides 


0 


6 


2 


2 


10 


24 


Terminology appropriate 


2 


5 


1 


2 


10 


25 


Awareness that outputs exist 


0 


0 


9 


0 


9 


02 


Quantity of outputs/data 


1 


0 


4 


2 


7 


28 


Operable by others 


0 


5 


2 


0 


7 


20 


Performs consistently 


2 


2 


1 


0 


5 


26 


Components are complementary 


0 


3 


2 


0 


5 


32 


Created by reputable producer 


0 


3 


0 


1 


4 


10 


Obvious (direct) termination 


3 


0 


0 


0 


3 


29 


Meets design expectations 


1 


0 


1 


1 


3 


03 


Quantity of effort expended 


0 


1 


0 


1 


2 


27 


Good physical repair/quality 


0 


0 


1 


0 


1 


31 


Meets legal constraints 


0 


1 


0 


0 


1 


33 


Inclusions are representative 


0 


0 


1 


0 


1 


'otals 




93 


358 


217 


107 


775 



O 

ERIC 
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APPENDIX 5 



(Continued) 





Primary Standard Category 




Project Focus 




Set 




Res 


Dev Diff Eval 


Totals 


LM 


Process Standards 


(5) 


(7) (5) (3) 


(20) 



Coding 



34 


Impact of effort favorable 


3 


5 


16 


10 


34 


17 


External enthusiasm evident 


6 


4 


20 


2 


32 


04 


Deadlines are met 


5 


9 


9 


6 


29 


13 


Work conducted w/in budget 


2 


7 


18 


1 


28 


36 


Employment criteria met 


1 


6 


17 


0 


24 


15 


Tasks perceived and acted upon 


0 


2 


16 


0 


18 


05 


Acceptable level of output 


3 


6 


3 


3 


15 


26 


Values and objectives match 


2 


4 


9 


0 


15 


24 


Costs acceptable for benefits 


0 


5 


7 


0 


12 


07 


An expected activity occurs 


3 


1 


3 


4 


11 


14 


No felt deficiencies 


1 


5 


5 


0 


11 


20 


Performance respected 


1 


4 


4 


2 


11 


11 


Costs consistent w/estimates 


3 


4 


3 


0 


10 


16 


External cooperation gained 


2 


4 


3 


1 


10 


01 


Personnel cooperate 


3 


1 


4 


1 


9 


02 


Personnel are satisfied 


0 


4 


4 


1 


9 


25 


Staff reflect trust 


0 


2 


3 


2 


7 


12 


No obvious omissions 


1 


0 


2 


3 


6 


28 


Closure reached on questions 


0 


2 


2 


2 


6 


32 


Resources available on request 


0 


0 


1 


5 


6 


35 


Output distributed/requeste * 


0 


0 


2 


4 


6 


09 


Outside contributions accepted 


2 


3 


0 


0 


5 


18 


Desired personnel obtained 


1 


1 


2 


1 


5 


33 


Personal growth/productivity 


n 


0 


4 


1 


5 


03 


Minimum correction required 


0 


0 


3 


1 


4 


10 


Maximum possible participation 


2 


1 


1 


0 


4 


30 


Personnel loss not excessive 


1 


0 


3 


0 


4 


41 


Guidelines are followed 


0 


1 


2 


1 


4 


08 


Staff contributions accepted 


0 


2 


1 


0 


3 


21 


Follow-on proposals are funded 


0 


2 


1 


0 


3 


22 


Feedback occurs 


0 


2 


1 


0 


3 


27 


Decisions result in action 


0 


2 


1 


0 


3 


29 


Creativity in work evidenced 


0 


0 


3 


0 


3 


31 


Overtime worked voluntarily 


0 


1 


2 


0 


3 


06 


Work structure is efficient 


0 


0 


2 


0 


2 



O 

ERIC 
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APPENDIX 5 



(Continued) 





Primary Standard Category 


Project Focus 


Totals 

(20) 


Set 

L.M 

Coding 


Process Standards 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


23 


Project view accepted 


0 


2 


0 


0 


2 


42 


Revision yields improvement 


0 


2 


0 


0 


2 


19 


Adequate reputation w/sponsor 


0 


1 


0 


0 


1 


37 


Staff adaptive to situations 


0 


0 


1 


0 


1 


38 


Equity in opportunity to input 


1 


0 


0 


0 


1 


39 


Evidence of pre-planning 


0 


0 


1 


0 


1 


40 


Output published externally 


0 


0 


0 


1 


1 


43 


Policy constraints minimal 


0 


0 


0 


1 


1 


44 


Responsibility taken willingly 


0 


0 


1 


0 


1 


45 


Resources used efficiently 


0 


0 


1 


0 


1 


46 


Staff are self-directing 


0 


0 


1 


0 


1 



Totals 


4 3 


95 


182 


53 


373 


Grand Totals 


136 


453 


399 


160 


1148 



3 




o 

ERIC 
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APPENDIX 6 



Primary Categories of Tasks, Organized Within Cluster 
Categories, and the Number of Task Statements 
Coded Within Them by Project Focus 



Set 




Project Focus 




NO QR 
Coding 


Tasks 


Res 

(5) 


Dev Diff Eval 
(7) (5) (3) 


Totals 

(20) 



TASK CLUSTER 



04 


Producing the output 


43 175 158 


98 


47^ 


PRIMARY TASK CATEGORIES 



04 


02 


Construct/assemble components of output. 


17 


97 


75 


36 


225 


04 


05 


Produce output. 


7 


31 


29 


10 


77 


04 


13 


Construe t/employ production tools/aids. 


5 


10 


17 


10 


42 


04 


01 


Construct/assemble measurement tools. 


5 


7 


10 


14 


36 


04 


14 


Adapt materials for local/specific use. 


1 


4 


9 


8 


22 


04 


08 


Select/produce data processing/handling. 


2 


1 


3 


8 


14 


04 


04 


Secure/select required materials/comps. 


2 


7 


2 


2 


13 


04 


07 


Package output. 


2 


3 


1 


2 


8 


04 


15 


Copy output for other media applications. 


0 


8 


0 


0 


8 


04 


12 


Invent new production method /instrument . 


0 


2 


4 


0 


6 


04 


16 


Coach talent (filming, video, sound). 


0 


1 


4 


0 


5 


04 


18 


Provide materials to producers. 


0 


0 


3 


2 


5 


04 


09 


Translate outcomes/content to user terms. 


0 


1 


0 


3 


4 


04 


17 


Organize presentation components. 


0 


3 


1 


0 


4 


04 


19 


Adapt content into processing format. 


0 


0 


0 


3 


3 


04 


11 


Get permission to use copyright material. 


2 


0 


0 


0 


2 


TASK CLUSTER 


05 




Collecoxng/proce88ing data 


71 


106 


102 


63 


342 



PRIMARY TASK CATEGORIES 



05 02 Administer/ Implement data instruments. 





12 22 



37 



8 



79 



APPENDIX 6 



(Continued) 



Set 
NO QR 
Cod ing 


j Tasks 

i 


Project Focus 
I Res Dev Diff Eval ! 
! (5) (7) (5) (3) 


Totals 

(20) 


05 


03 


Record/score/classify/code raw data. 


8 


21 


17 


11 


57 


05 


09 


Interpret data. 


10 


11 


18 


11 


j0 


05 


11 


Try out output/component for operability. 


8 


14 


13 


5 


40 


05 


06 


Perforin analyses as specif ied/indicated . 


8 


14 


5 


6 


33 


05 


01 


Conduct/operate program to be measured. 


3 


10 


5 


0 


18 


05 


04 


Transfer raw data into storage form. 


3 


4 


5 


4 


16 


05 


07 


Compile computational data. 


0 


5 


1 


6 


12 


05 


05 


Translate data into processing form 


5 


1 


0 


5 


11 


05 


08 


Prepare supporting documentation. 


4 


3 


0 


? 


9 


05 


14 


Specify parameters of sub-routines. 


3 


0 


0 


i 


4 


05 


12 


Specify job control lang for computer. 


3 


0 


0 


0 


3 


05 


18 


Construct/maintain data displays 


0 


0 


0 


3 


3 


05 


10 


Acquire artifacts. 


0 


1 


1 


0 


2 


05 


13 


Identify sub-routines req by computer. 


1 


0 


0 


0 


1 


05 


15 


Sequence the computer program. 


1 


0 


0 


0 


1 


05 


16 


Determine constants inherent in analysis. 


1 


0 


0 


0 


1 


05 


17 


Specify format for computer printout. 


1 


0 


0 


0 


1 


05 


19 


Specify/identify data identity codes. 


0 


0 


0 


1 


1 



TASK CLUSTER 



06 Assessing the output quality 32 178 91 35 336 



PRIMARY TASK CATEGORIES 



06 


14 


Note/make advisable modif ication(s) . 


7 


90 


47 


14 


158 


06 


15 


Proofread/edit written material. 


2 


26 


o 


8 


45 


06 


01 


Check output against specifications. 


6 


22 


13 


1 


42 


06 


08 


Determine inadequate output components. 


6 


13 


7 


1 


27 


06 


07 


Judge output effectiveness/consistency. 


1 


3 


7 


0 


11 


06 


06 


Judge output operability/acceptability. 


3 


3 


2 


1 


9 


06 


10 


Confirm operability/validity of tools. 


0 


4 


0 


5 


9 


06 


11 


Judge effectiveness of data gathering 


1 


3 


2 


1 


7 


06 


02 


Judge significance of performance result. 


4 


1 


0 


0 


5 


06 


19 


Edit video-audio-film materials. 


0 


3 


2 


0 


5 



APPENDIX 6 



(Continued) 



Set 
NO QR 
Coding 


Tasks 

. i 




Project Focus 


Totals 

(20) 


Res 

(5) 


Dev 

(7) 


Dlff 

(1)_ 


Eval 1 

-13) ' 


06 


05 


Identify une/pected/incongruent outcomes 


0 


2 


1 


1 


4 


06 


12 


Relate data to decision situations. 


0 


3 


0 


1 


4 


06 


16 


Determine adeq of implementation process. 


2 


2 


0 


0 


4 


06 


17 


Check recommended changes specs. 


0 


2 


0 


0 


2 


06 


20 


Select best alternative method/procedure. 


0 


0 


0 


2 


2 


06 


03 


Determine achievement of objectives. 


0 


0 


1 


0 


1 


06 


18 


Judge output against personal standards. 


0 


1 


0 


0 


1 



TASK CLUSTER 



22 




Effecting accountability 


44 


102 


120 


48 


314 






PRIMARY TASK CATEGORIES 












22 


04 


Establish/maintain time lines. 


9 


17 


7 


13 


46 


22 


02 


Delegate responsibilities /authority. 


7 


12 


18 


2 


39 


22 


12 


Explicate procedures/strategies/duties. 


2 


5 


19 


1 


27 


22 


14 


Issue production orders. 


2 


15 


3 


3 


23 


22 


13 


Interact with advisory services. 


t 

J 


7 


13 


0 


21 


22 


05 


Project cost estimates. 


2 


11 


4 


3 


20 


22 


07 


Negotiate/ renegotiate funding/objectives. 


3 


2 


3 


7 


15 


22 


09 


Monitor expenditure of t ime/sves /money . 


6 


4 


5 


0 


15 


22 


10 


Report status as required/appropriate. 


0 


3 


10 


1 


14 


22 


19 


Maintain materials/use/f ideal records. 


3 


3 


4 


4 


14 


22 


03 


Determine/verif y/monitor budget balances . 


1 


3 


5 


0 


9 


22 


11 


Explicate obj/outputs to be produced. 


1 


3 


4 


1 


9 


22 


15 


Prepare detailed spending plan. 


0 


4 


4 


0 


8 


22 


08 


Approve disbursement of funds. 


4 


2 


1 


0 


7 


22 


18 


Insure optimal use of resources. 


0 


0 


4 


2 


6 


22 


21 


Develop record keeping system. 


1 


1 


3 


1 


6 


22 


17 


Maintain duplicate records/files. 


0 


1 


3 


1 


5 


22 


01 


Specify accountability procedures. 


0 


1 


2 


1 


4 


22 


22 


Es t ima t e / p r ovid e mater ial / personnel . 


1 


0 


1 


1 


3 


22 


28 


Specify /establish/adopt policies. 


0 


1 


0 


2 


3 



0 
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APPENDIX 6 



(Continued) 



Set 
NO QR 
Coding 


Tas<ts 


Project Focus 


Totals 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3)_ 


22 


29 


Oversee/monitor adherence to requiremts. 


0 


2 


0 


1 


3 


22 


06 


Determine nature/content of reporting. 


1 


1 


0 


0 


2 


22 


16 


Compare expenditure rate with plan 


0 


1 


1 


0 


2 


22 


20 


Develop/maintain inventory system. 


0 


0 


2 


0 


2 


22 


25 


Pay bills/cdllect payments. 


0 


1 


1 


0 


2 


22 


26 


Divide work into appropriate segments. 


0 


0 


1 


1 


2 


22 


27 


Monitor /determine cost-effectiveness. 


0 


0 


2 


0 


2 


22 


73 


Direct/x'equest special reports. 


0 


1 


0 


0 


1 


22 


24- 


Determine/comply w/sponsor f s procedures. 


0 


1 


0 


0 


1 


22 


30 


Obtain staff input regarding time/funds. 


0 


0 


0 


1 


1 


22 


31 


Participate in production efforts as req. 


0 


0 


0 


1 


1 


22 


32 


Approve budget. 


0 


0 


0 


1 


1 


TASK CLUSTER 



03 




Designing the output 


30 . 


118 


88 


64 


300 






PRIMARY TASK CATEGORIES 












03 


10 


Specify/identify character of materials. 


4 


43 


21 


9 


77 


03 


06 


Specify treatments /procedures/ strategies . 


6 


11 


15 


7 


39 


03 


09 


Visualize/ specify format /appearance. 


2 


13 


10 


3 


28 


03 


03 


Determine relation of variables/compnts . 


4 


14 


1 


6 


25 


03 


08 


Translate objectives/data into meaning. 


0 


3 


18 


0 


21 


03 


05 


Specify/select grouping/ sampling. 


3 


2 


1 


12 


18 


03 


04 


Select/specify populacions (s) involved . 


2 


1 


8 


2 


13 


03 


12 


Estab tolerances/ effectiveness criteria . 


0 


5 


4 


3 


12 


03 


23 


Consider/design alternative approaches. 


0 


3 


4 


4 


11 


03 


20 


State overall design. 


1 


5 


3 


0 


9 


03 


07 


Conceptualize output application 


3 


4 


0 


1 


8 


03 


01 


Identify decision situations of interest. 


0 


2 


0 


5 


7 


03 


02 


Isolate contaminating variables. 


2 


1 


1 


3 


7 


03 


18 


Specify data/info processing/up-date. 


1 


2 


0 


4 


7 


03 


13 


Establish specs for output try-out/demo 


0 


5 


0 


0 


5 



O 

ERIC 
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APPENDIX 6 



(Continued ) 



Set 
NO QR 
Coding i 


Tasks 




Project 


Focus i 


Totals 

(20) 


Res 

(5) 


Dev Diff 
(7) (5) 


Eval ■ 
(3) , 


03 


17 


Specify measures to be used. 


0 


3 


i 


1 


5 


03 


19 


Specify kind of data analysis to be used. 


0 


0 


i 


2 


3 


03 


22 


Specify data collection schedule. 


0 


1 


0 


2 


3 


03 


15 


Specify site adaptations required. 


1 


0 


0 


0 


1 


03 


24 


Visualize/Specify effect of output. 


1 


0 


0 


0 


1 


TASK CLUSTER 



24 




Effecting quality control 


19 


101 


134 


44 


298 






PRIMARY TASK CATEGORIES 












24 


11 


Proof /critique produc ts /performances 


2 


16 


34 


9 


61 


24 


17 


Direct/request critques from others. 


4 


32 


11 


2 


49 


24 


10 


Monitor progress/status of work. 


3 


4 


22 


7 


36 


24 


09 


Initiate/direct quality control action 


3 


6 


17 


7 


33 


24 


07 


Monitor personnel performance 


3 


8 


8 


5 


24 


24 


06 


Monitor adherence to output specs. 


0 


8 


12 


2 


22 


24 


04 


Specify/develop improvement procedures . 


0 


6 


5 


2 


13 


24 


16 


Negotiate quality issues with sponsor. 


0 


7 


4 


0 


11 


24 


23 


Confirm accuracy of data imputs/cutputs . 


1 


0 


0 


7 


8 


24 


14 


Determine need for /present alternatives. 


2 


2 


2 


1 


7 


24 


01 


Specify /recommend criteria for output. 


0 


3 


3 


0 


6 


24 


21 


Adjust staff for operation compatibility. 


0 


0 


4 


0 


4 


24 


22 


Select outputs for further review. 


0 


2 


2 


0 


4 


24 


20 


Monitor philosophical consistency of opn. 


0 


0 


3 


0 


3 


24 


08 


Monitor training effectiveness. 


0 


1 


0 


1 


2 


24 


12 


Monitor adherence to policies. 


0 


1 


0 


1 


2 


24 


13 


Monitor decisions made. 


1 


0 


1 


0 


2 


24 


15 


Identify quality control issues. 


0 


0 


2 


0 


2 


24 


18 


Monitor staff awareness of responsbility . 


0 


2 


0 


0 


2 


24 


19 


Approve potential staff for employment. 


0 


0 


2 


0 


2 


24 


24 


Monitor/judge project impact. 


0 


1 


1 


0 


2 


24 


03 


Specify standards . , for trng effectiveness 


0 


1 


0 


0 


1 


24 


05 


Delegate quality control responsibility. 


0 


1 


0 


0 


1 


24 


25 


Elicit author approval re project changes. 


0 


0 


1 


0 


1 
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APPENDIX 6 



(Continued) 



Set 
NO QR 
Coding 


n — 

1 

| Tasks 


Project Focus 


Total s 
(20) 


Res 

l(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


TASK CLUSTER 


01 




Clarifying Problem Addressed 


32 


124 


105 


36 


297 






PRIMARY TASK CATEGORIES 












01 


12 


Review cf /familiarization w/materials 


12 


42 


44 


12 


110 


01 


01 


Review problem-relevant literature/info . 


7 


30 


7 


4 


48 


01 


11 


Visualize application of technique/ideas. 


0 


7 


17 


3 


27 


01 


02 


Specify context /scope/limits of effort. 


1 


9 


11 


2 


23 


01 


06 


Observe relevant /related field opns. 


0 


3 


12 


0 


15 


01 


07 


Identify characteristics tgt population. 


1 


5 


5 


4 


15 


01 


06 


Determine problems to be addressed. 


5 


7 


1 


1 


14 


01 


03 


Specify variables to be studied. 


4 


1 


1 


5 


11 


01 


13 


Obtain literature relevant to problem. 


0 


5 


2 


0 


7 


01 


04 


Identify contaminating variables. 


1 


1 


1 


3 


6 


01 


10 


Determine priorities of problems. 


0 


2 


3 


1 


6 


01 


15 


Operationally define terms/variables . 


0 


5 


0 


1 


6 


01 


09 


Determine sensitive areas in tgt context. 


1 


1 


1 


0 


3 


01 


14 


Compare existing frameworks/procedures . 


0 


3 


0 


0 


3 


01 


05 


State major hypo theses /philos position. 


0 


2 


0 


0 


2 


01 


16 


Identify/assess divergent points of view. 


0 


1 


0 


0 


1 


TASK CLUSTER 


29 




Facilitating relationships 


32 


76 


69 


48 


225 






PRIMARY TASK CATEGORIES 












29 


14 


Interact with setting to collaborate. 


9 


9 


25 


9 


52 


29 


15 


Interact w/setting to enhance. 


8 


5 


19 


11 


43 


29 


16 


Make materials/experiences available. 


1 


12 


6 


12 


31 


29 


04 


Interact w/setting to elicit cooperation. 


5 


9 


5 


5 


24 


29 


05 


Communicate behavior required of setting. 


3 


10 


1 


5 


19 



APPENDIX 6 



(Continued) 



Set 
NO QR 
Coding 


Tasks j 


Res 

(5) 


Project Focus i 

Dev Diff Eval I 
(7) (5) (3) ! 


Totals 

(20) 


29 


06 


Explain/prepare rationale for proc to site. 


1 


11 


1 


0 


13 


29 


10 


Promote or maintain focus of effort. 


0 


5 


3 


3 


11 


29 


13 


Maintain contacts with field. 


2 


2 


5 


1 


10 


29 


01 


Monitor field setting views/concerns. 


1 


4 


0 


0 


5 


29 


09 


Adapt to accommodate differences. 


0 


3 


1 


0 


4 


29 


07 


Negotiate resolution of differences. 


0 


2 


1 


0 


3 


29 


08 


Locate/document sources of disagreement. 


0 


2 


0 


0 


2 


29 


11 


Translate language into understood terms. 


0 


1 


1 


0 


2 


29 


12 


Provide for alternate output treatment 


0 


1 


1 


0 


2 


29 


02 


Give visibility to cooperating people. 


1 


0 


0 


0 


1 


29 


03 


Give visibility to cooperating agencies. 


1 


0 


0 


0 


1 


29 


17 


Provide foreign language translations. 


0 


0 


0 


1 


1 


29 


18 


Endorsement/support for field personnel. 


0 


0 


0 


1 


1 



TASK CLUSTER 



31 Diffusing info within project 



34 61 78 32 205 



PRIMARY TASK CATEGORIES 



31 


04 


31 


06 


31 


12 


31 


01 


31 


08 


31 


09 


31 


03 


31 


02 


31 


15 


31 


05 


31 


11 


31 


10 


31 


13 


31 


07 


31 


14 



Lead/participate in task-oriented groups. 
Receive and react to relevant input. 

Refer problem to appropriate specialists. 
Plan/conduct staff meetings/conferences 
Communicate production guidelines. 

Keep involved personnel informed generally. 
Produce/circulate bulletins/memos/news . 

Make formal presentations. 

Advise technically within expertise. 

Join in/lead info meetings/contacts. 

Advise on admin concerns w/in expertise. 
Communicate relevant experience info. 

Record events (e.g., minutes, processes). 
Communicate decisions made. 

Serve in consulting capacity to project. 



6 14 

4 7 

3 5 

0 7 

0 6 

2 8 

5 5 

3 3 

2 1 

5 1 

1 0 

0 2 

3 0 

0 1 

0 1 



12 

25 

9 

7 

11 

4 

1 

2 

0 

2 

3 

1 

0 

1 

0 



8 

1 

2 

4 

1 

4 

3 

2 

6 

0 

1 

0 

0 

0 

0 



40 

37 

19 

18 

18 

18 

14 

10 

9 

8 

5 

3 

3 

2 

1 






APPENDIX 6 



(Continued) 



Set 




, 


Project Focus 


— 


NO QR 


Tasks 


Res 


Dev Diff Eval 


Totals 


Coding 




1 .( 5 ) 


(7) (5) (3) 


(20) 



TASK CLUSTER 



21 




Procuring professional staff 


21 


61 


83 


36 


201 






PRIMARY TASK CATEGORIES 












21 


11 


Provide training as appropriate. 


2 


13 


17 


12 


44 


21 


10 


Select/hire/appoint personnel . 


6 


4 


15 


8 


33 


21 


09 


Interview/screen candidates. 


1 


11 


16 


4 


32 


21 


08 


Identify potential candidates. 


0 


11 


4 


2 


17 


21 


01 


Analyze work for competencies required. 


4 


4 


4 


3 


15 


21 


12 


Negotiate w/candidate to employ/ part ic. 


3 


6 


3 


1 


13 


21 


03 


Formulate job descr iptions/qualif /salary . 


2 


2 


5 


0 


9 


21 


07 


Make vacancies known. 


0 


2 


4 


3 


9 


21 


13 


Determine manpower needs. 


1 


2 


4 


1 


8 


21 


02 


Determine competencies available. 


0 


1 


4 


2 


7 


21 


15 


Recommend personnel for employment. 


0 


1 


2 


0 


3 


21 


17 


Determine/assess best staffing pattern. 


1 


0 


2 


0 


3 


21 


05 


Determine need for other points of view. 


0 


2 


0 


0 


2 


21 


06 


Maintain list of replacement personnel. 


0 


1 


0 


0 


1 


21 


14 


Determine tasks to procure staff. 


1 


0 


0 


0 


1 


21 


16 


Obtain approval to hire. 


0 


1 


0 


0 


1 


21 


18 


Obtain/assess candidate's goals. 


0 


0 


1 


0 


1 


21 


19 


Review opns relative to existing staff. 


0 


0 


1 


0 


1 


21 


20 


Confer w/others re candidate selection. 


0 


0 


1 


0 


1 


TASK CLUSTER 


23 




Procuring systems/services 


30 


45 


117 


7 


199 



PRIMARY TASK CATEGORIES 



23 


05 


Negotiate contract/coop w/sites/services . 


7 


22 


63 


3 


95 


23 


03 


Identify/ study possible sites/services. 


6 


5 


33 


0 


44 


23 


04 


Select /obtain sites /facilities /services. 


3 


12 


11 


0 


26 


23 


07 


Arrange visitation/^fcheflule as needed. 


6 


0 


5 


3 


14 


23 


02 


Determine facilitiwVrfeld material req. 


3 


1 


2 


0 


6 
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APPENDIX 6 



(Continued) 



Set 






Project Focus j 




NO 


QR 


Tasks 


Res 


Dev Diff 


Eval ! 


Totals 


Coding 




(5) 


(7) (5) 


(3) 


(20) 


23 


08 


Make public service/material/site needs 


2 


2 1 


0 


5 


23 


01 


Determine tasks/services to be performed. 


2 


1 1 


0 


4 


23 


09 


Procure required tools/equipment. 


1 


2 1 


0 


4 


23 


06 


Get permission to use copyright material. 


0 


0 0 


1 


1 



TASK CLUSTER 



02 




Formulating objectives 


15 


67 


35 


23 


140 






PRIMARY TASK CATEGORIES 












02 


07 


Confer with colleagues/others re objects. 


11 


35 


16 


15 


77 


02 


02 


Conceptualize the product and context. 


2 


14 


8 


3 


27 


02 


06 


Create objectives. 


0 


6 


8 


2 


16 


02 


03 


Formulate specific questions/hypotheses . 


0 


4 


2 


1 


7 


02 


01 


Conceptualize the theoretical framework. 


1 


5 


0 


0 


6 


02 


05 


State problem in context of theory. 


1 


3 


0 


1 


5 


02 


04 


Determine nature of msmt objectives. 


0 


0 


1 


1 


2 



TASK CLUSTER 



25 Maintaining job satisfaction 0 18 98 8 124 



PRIMARY TASK CATEGORIES 



25 


01 


Specify/explain agency/project position. 


0 


2 


63 


4 


69 


25 


02 


Provide means for handling staff views. 


0 


5 


7 


1 


13 


25 


09 


Encourage optimal self-direction. 


0 


2 


6 


1 


9 


25 


04 


Counsel with/be supportive of personnel. 


0 


3 


4 


0 


7 


25 


03 


Recognize merit. 


0 


0 


4 


1 


:> 


25 


08 


Reconcile differences of opinion. 


0 


1 


3 


0 


4 


25 


05 


Adjust tasks to fit interests/limits. 


0 


0 


2 


1 


3 
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APPENDIX 6 



(Continued) 



Set 
NO QR 
Coding 


j 

■ Tasks 

i 


-T 

1 

i Res 

(5) 


Project Focus j 

Dev Diff Eval j 
(7) (5) (3) ! 


Total s 
(20) 


25 


13 


Provide for social activities 


0 


1 


2 


0 


3 


25 


06 


Recommend promotions/ salary increases . 


0 


1 


1 


0 


2 


25 


07 


Encourage creativity. 


0 


0 


2 


0 


2 


25 


10 


Foster mutual respect. 


0 


0 


2 


0 


2 


25 


11 


Support staff in decisions they make. 


0 


1 


1 


0 


2 


25 


12 


Foster feelings of security in job. 


0 


1 


0 


0 


1 


25 


14 


Act on behalf of staff in agency matters. 


0 


1 


0 


0 


1 


25 


15 


Nurture staff feelings of importance. 


0 


0 


1 


0 


1 


TASK CLUSTER 



33 




Effecting decision mechanisms 


ii 


11 


33 


12 


67 






PRIMARY TASK CATEGORIES 












33 


03 


Make decisions. 


9 


8 


9 


9 


35 


33 


07 


Participate in decision-making process. 


2 


1 


8 


1 


12 


33 


02 


Specify kinds of decisions at each level. 


0 


0 


3 


2 


5 


33 


10 


Establish organizational structure. 


0 


0 


4 


0 


4 


33 


06 


Specify vehicles for decision-making. 


0 


0 


3 


0 


3 


33 


08 


Identify/specify sources of problems 


0 


0 


3 


0 


3 


33 


11 


Get partic in decisions-conceptualizing . 


0 


1 


1 


0 


2 


33 


01 


Specify/determine key decision points. 


0 


0 


1 


0 


1 


33 


04 


Specify who needs what kinds of info. 


0 


1 


0 


0 


1 


33 


09 


Request support for decisions made. 


0 


0 


1 


0 


1 



TASK CLUSTER 



30 




Effecting info flow patterns 


4 


18 


29 


10 


61 






PRIMARY TASK CATEGORIES 












30 


03 


Insure that key decision points get info. 


3 


10 


16 


4 


33 


30 


02 


Determina/use information sources. 


0 


4 


6 


1 


11 


30 


01 


Define project information flow policies. 


0 


0 


2 


3 


5 


30 


07 


Explicate/ employ ways of enhancing flow. 


1 


2 


1 


1 


5 
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APPENDIX 6 



(Continued ) 



Set 
NO QR 
Coding 


Tasks 


Project Focus 
Res Dev Diff Eval 
(5) (7) (5) (3) 


Totals 

(20) 


30 04 


Provide communications equip/capability 


0 


1 




1 


1 


3 


30 06 


Monitor information flow. 


0 


1 




1 


0 


2 


30 05 


Define lines of communications. 


0 


0 




1 


0 


1 


30 08 


Coordinate people interactions. 


0 


0 




1 


0 


1 



TASK CLUSTER 



26 




Facilitating growth of staff 


3 


7 


26 


11 


47 






PRIMARY TASK CATEGORIES 












26 


01 


Provide for guidance within project. 


3 


7 


13 


7 


30 


2b 


02 


Provide opportunities for growth. 


0 


0 


5 


1 


6 


26 


05 


Provide opportunities for exposure. 


0 


0 


4 


0 


4 


26 


03 


Provide incentive for quality work. 


0 


0 


3 


0 


3 


26 


06 


Specify ownership/authorship policies. 


0 


0 


1 


1 


2 


26 


07 


Encourage publication of important work. 


0 


0 


0 


2 


2 


TASK CLUSTER 


32 




Diffusing info beyond project 


10 


9 


15 


6 


40 






PRIMARY TASK CATEGORIES 













32 


04 


Report project at agency/other meetings. 


2 


6 


4 


4 


16 


32 


02 


Prepare/disseminate promotional inf o . 


2 


1 


2 


0 


5 


32 


07 


Provide info to dissemination services. 


2 


0 


2 


0 


4 


32 


01 


Write professional papers. 


3 


0 


0 


0 


3 


32 


08 


Interact with others regarding promotion. 


3. 


2 


0 


0 


3 


32 


10 


Insure appropriate visibility of project 


0 


0 


2 


1 


3 


32 


06 


Interact with dissemination servies. 


0 


0 


2 


0 


2 


32 


03 


Act as consultant. 


0 


0 


1 


0 


1 


32 


09 


Enlist community help in dissemination. 


0 


0 


1 


0 


1 


32 


11 


Approve articles/papers for publication. 


0 


0 


0 


1 


1 


32 


12 


Prepare project descriptive materials. 


0 


0 


1 


0 


1 
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APPENDIX 6 



(Continued) 



Set 




Project Focus 




NO QR 

Coding 


Tasks 


Res Dev Diff Eval 
(5) (7) (5) (3) 


Totals 

(20) 



TASK CLUSTER 



07 


Diffusing the output 


i ii 


20 


4 


36 


PRIMARY TASK CATEGORIES 



07 


17 


Distribute output/component. 


0 


2 


8 


3 


13 


07 


14 


Negotiate user acceptance/use. 


0 


5 


1 


0 


6 


07 


15 


Provide for user training as required. 


0 


2 


3 


0 


5 


07 


09 


Prepare/write promotional materials 


1 


1 


1 


0 


3 


07 


01 


Study/describe characteristics of output. 


0 


1 


0 


1 


2 


07 


12 


Present/demonstrate output to user* 


0 


0 


2 


0 


2 


07 


13 


Negotiate for user try-out. 


0 


0 


2 


0 


2 


07 


08 


Negotiate for sale of copyrighted matter. 


0 


0 


1 


0 


1 


07 


11 


Disseminate promotional materials. 


0 


0 


1 


0 


1 


07 


18 


Maintain distribution lists. 


0 


0 


1 


0 


1 



TASK CLUSTER 


27 




Enhancing physical environment 


0 


0 


8 


0 


8 






PRIMARY TASK CATEGORIES 












27 


06 


Optimize accessibility to equip/services 


0 


0 


3 


0 


3 


27 


01 


Provide optimal proximity of staff. 


0 


0 


2 


0 


2 


27 


04 


Provide meeting/relaxation facility. 


0 


0 


2 


0 


2 


27 


03 


Provide for privacy. 


0 


0 


1 


0 


1 
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APPENDIX 6 



(Continued ) 



Set 




1 Project Focus 




NO QR 
Coding 


Tasks 


j Res Dev Diff Eval 

! (5) (7) (5) (3) 

1 _ ! 


Totals 

(20) 

1 


TASK CLUSTER 


28 


Maintaining equity among staff 


112 2 


6 



PRIMARY TASK CATEGORIES 



28 


01 


Set demands in keeping with competencies. 


i 


0 


1 


1 


3 


28 


02 


Determine excessive/low work loads. 


0 


1 


1 


0 


2 


28 


03 


Insure equity in application of rules. 


0 


0 


0 


1 


1 
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APPENDIX 7 



Primary Categories of Knowledge, and the Number of 
Knowledge Statements Coded Within 
Them by Project Focus 



Set 

UV 

Coding 


Enabling Knowledges 

i 

i 


Project Focus 


Totals 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

.(5) 


Eval 

(3) 


06 


Project variables: external 


29 


38 


81 


9 


157 


03 


Subjects related to RDD&E 


38 


34 


22 


50 


144 


04 


Technical/ professional topics 


14 


56 


44 


22 


136 


08 


Project operation: specific 


12 


40 


36 


8 


96 


07 


Project operation: general 


7 


22 


19 


5 


53 


05 


Project focus topics, external 




15 


15 


9 


50 


02 


Subjects learned in courses 


3 


16 


9 


13 


41 


12 


Resources : Personnel 


4 


22 


7 


5 


38 


24 


Process implementation (proj ) 


3 


6 


17 


11 


37 


22 


Use of equipment /systems 


3 


12 


9 


6 


30 


01 


Standard school subjects 


0 


16 


8 


2 


26 


09 


Scheduling & organizing 


2 


9 


13 


1 


25 


11 


Fiscal matters 


6 


5 


4 


4 


19 


10 


Staff status/ responsibilities 


1 


3 


12 


1 


17 


18 


Staff competencies/ interest s 


0 


3 


8 


4 


15 


19 


Technical terminology /language 


0 


5 


7 


1 


13 


23 


Char's of target audience 


4 


5 


0 


4 


13 


17 


Writing styles 


1 


8 


0 


1 


10 


16 


Guidelines for reporting 


1 


2 


4 


1 


8 


25 


Sources of info/materials 


1 


1 


4 


2 


8 


21 


Management techniques 


2 


1 


2 


2 


7 


28 


Group dyn/decision processes 


0 


1 


3 


2 


6 


27 


Potential field settings 


0 


4 


0 


1 


5 


13 


Resources: money 


2 


1 


0 


0 


3 


14 


Resources: time 


2 


0 


0 


0 


2 


26 


Incorrect data entries/posting 


2 


0 


0 


0 


2 


20 


Sponsor concerns 


0 


0 


1 


0 


1 


29 


Need for mater ial/ inf o 


0 


0 


1 


0 


1 


30 


Errors in strategies/ judgment 


0 


0 


0 


1 


1 


Totals 


148 


325 


326 


165 


964 






o 

EKIC 




APPENDIX 8 



Primary Categories of Skills, and the Number of 
Skill Statements Coded Within 
Them by Project Focus 





Set 




Project Focus 




uv 


Enabling Skills 


Res 


Dev 


Diff 


Eval 


Totals 


Coding 




(5) 


(7) 


(5) 


(3) 


(20) 


14 


Writ ing 


12 


35 


32 


11 


90 


02 


Facilitating people interactions 


7 


26 


28 


13 


74 


18 


Finding fits/ Integra t ing 


8 


30 


18 


8 


64 


19 


Planning/ conceptualizing 


6 


19 


11 


9 


45 


11 


Disciplining self 


5 


17 


7 


9 


38 


10 


Analytical data handling 


7 


6 


13 


8 


34 


23 


Persuading/ justify ing 


4 


7 


19 


3 


33 


05 


Programming project events 


3 


12 


9 


6 


30 


01 


Teaching 


4 


11 


9 


3 


27 


17 


Interpreting language 


3 


12 


7 


5 


27 


25 


Applying measurement tools 


2 


5 


12 


7 


26 


27 


Using equipment/systems 


3 


5 


10 


8 


26 


09 


Analytical problem solving 


1 


10 


2 


10 


23 


20 


Exercising judgment 


1 


12 


9 


1 


23 


35 


Communicating clearly 


1 


11 


2 


8 


22 


29 


Getting others to perform 


1 


4 


9 


4 


18 


06 


Programming subject matter 


1 


13 


2 


1 


17 


24 


Explicating goals/procedures 


1 


7 


7 


2 


17 


30 


Adapting to situation/demands 


2 


4 


9 


2 


17 


15 


Presenting orally 


3 


8 


5 


0 


16 


08 


Analytical reading/ study 


0 


7 


5 


2 


14 


22 


Estimating expenses/resources 


4 


6 


3 


1 


14 


26 


Locating/raaintaining info 


2 


8 


3 


1 


14 


37 


Assessing skill s/po ten t ials 


0 


6 


4 


2 


12 


39 


Constructing measurement tools 


2 


3 


2 


5 


12 


21 


Tracking act ivi ties /goals 


0 


2 


6 


3 


11 


34 


Coordinating activities 


4 


3 


3 


1 


11 


38 


Using resources effectively 


0 


3 


o 

L 


5 


10 


31 


Taking another's perspective 


0 


4 


2 


3 


9 


16 


Using media 


1 


4 


3 


0 


8 


03 


Translating content to media 


0 


5 


1 


1 


7 


04 


Using/applying feedback 


0 


3 


3 


1 


7 


13 


Listening 


0 


3 


4 


0 


7 


32 


Identifying/ correcting errors 


0 


3 


3 


1 


7 


49 


Interacting productively 


0 


3 


3 


0 


6 



O 

ERIC 



3.79 



APPENDIX 8 



(Continued) 



* 


Set 




Project Focus 




uv 


Enabling Skills 


Res 


Dev 


Diff 


Eval 


Totals 


Coding 




(5) 


V) 


(5) 


(3) 


(20) 


07 


Programming technical equip 


0 


0 


0 


5 


5 


33 


Graphically illustrating 


0 


2 


2 


0 


4 


42 


Eliciting responses 


0 


0 


3 


0 


3 


45 


Assessing personal performance 


0 


0 


1 


2 


3 


28 


Running task oriented meetings 


0 


0 


2 


0 


2 


36 


Cataloging/ classifying 


1 


0 


1 


0 


2 


40 


Defining terms/probleras 


0 


0 


0 


2 


2 


43 


Instilling confidence 


0 


0 


1 


1 


2 


44 


Fiscal accounting 


0 


0 


2 


0 


2 


46 


Placing others at ease 


0 


1 


1 


0 


2 


48 


Establishing credibility 


0 


1 


0 


1 


2 


12 


Disciplining others 


0 


1 


0 


0 


1 


41 


Maintaining physical condition 


0 


0 


1 


0 


1 


47 


Recalling 


0 


1 


0 


0 


1 


Totals 




89 


323 


281 


155 


848 



340 



334 



APPENDIX 9 



Primary Categories of Sensitivities, and the Number 
of Sensitivity Statements Coded Within 
Them by Project Focus 



Set 

UV 

Coding 


Enabling Sensitivities 


Project Focus 


Totals 

(20) 


Res 

(5) 


Dev 

(7) 


Diff 

(5) 


Eval 

(3) 


02 


Capabilities and limitations 


10 


39 


16 


11 


76 


16 


Existing value systems 


5 


29 


20 


10 


64 


03 


Needs of self and others 


2 


21 


23 


4 


50 


30 


Response sets of target audiences 


3 


8 


20 


7 


38 


22 


Responses of target audiences 


7 


12 


9 


3 


31 


01 


Values of self and others 


6 


11 


11 


1 


29 


04 


Interactions of self and others 


3 


6 


15 


1 


25 


13 


Language barriers 


2 


14 


6 


2 


24 


31 


Nature/scope of ou:put 


3 


8 


3 


5 


19 


18 


Potential conflict of interest 


4 


9 


4 


1 


18 


21 


Limitations of analyses/data 


2 


6 


4 


6 


18 


17 


Personality of others 


1 


5 


10 


1 


17 


09 


Awareness of alternatives 


4 


4 


0 


0 


8 


10 


/wareness of structure 


6 


5 


5 


0 


16 


27 


Acceptability of output 


1 


6 


7 


0 


14 


06 


Worth in disciplines/methods 


0 


5 


2 


6 


13 


47 


Willingness to work as needed 


0 


3 


1 


9 


13 


15 


Degrees of freedom to deviate 


2 


3 


5 


1 


11 


37 


Willingness to delegate 


0 


2 


3 


6 


11 


11 


Awareness of method 


3 


6 


1 


0 


10 


24 


Sources of error 


1 


3 


4 


2 


10 


20 


Unstated obligations 


2 


4 


2 


1 


9 


25 


Individual differences 


0 


2 


6 


1 


9 


34 


Willingness to accept guidance 


1 


3 


2 


3 


9 


19 


Supportiveness required 


0 


1 


5 


2 


8 


23 


Co st /bene fit factors 


1 


4 


3 


0 


8 


33 


Need for excellence in work 


0 


1 


3 


4 


8 


36 


Feeling of personal security 


1 


0 


3 


4 


8 


38 


Emerging directions 


2 


2 


1 


3 


8 


05 


Context of subject matter 


0 


2 


4 


1 


7 


14 


Reality in goal setting 


2 


5 


0 


0 


7 


28 


Admitting error/adapting 


0 


2 


1 


4 


7 


35 


Need to communicate effectively 


0 


2 


2 


3 


7 


44 


Interested in activities 


2 


0 


0 


5 


7 


26 


Recognition of data needs 


2 


1 


3 


0 


6 



O 

ERIC 



7 -6 
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APPENDIX 9 



(Continued) 



_L 





Set 




Project Focus 




uv 


Enabling Sensitivities 


Res 


Dev 


Diff 


Eval 


Totals 


Coding 


(5) 


(7) 


(5) 


P) 


(20) 


J 


Contractual/stated obligations 


0 


3 


0 


3 


6 


5 


Respect for/trust in others 


0 


0 


1 


5 


6 


29 


Willingness to experiment 


1 


3 


1 


0 


5 


07 


Context of objectives 


2 


2 


0 


0 


4 


42 


Enthusiasm 


0 


1 


0 


3 


4 


50 


Intellectual openness 


0 


2 


0 


2 


4 


52 


Awareness of staff affect 


0 


3 


1 


0 


4 


08 


Worth in objectives 


0 


2 


1 


0 


3 


12 


Role of catalyst/synthesizer 


0 


2 


1 


0 


3 


32 


Reality in spending 


0 


0 


3 


0 


3 


41 


Inquisitiveness 


0 


1 


1 


1 


3 


45 


Sound variability 


0 


0 


2 


0 


2 


48 


Common sense 


0 


1 


0 


1 


2 


51 


Possible points of confusion 


0 


2 


0 


0 


2 


53 


Creative 


0 


2 


0 


0 


2 


58 


Willingness to support staff 


0 


1 


1 


0 


2 


39 


Limits of "one-shot" efforts 


0 


1 


0 


0 


1 


43 


Challenged by activities 


0 


1 


0 


0 


1 


46 


Sense of humor 


0 


0 


1 


0 


1 


49 


Willingness to learn 


0 


1 


0 


0 


1 


54 


Manageability of data 


0 


1 


0 


0 


1 


56 


Habit of thoroughness 


0 


0 


0 


1 


1 


57 


Deadlines 


0 


1 


0 


0 


1 


Totals 




81 


264 


217 


123 


685 



336 




* 



APPENDIX 10 



The Functions Served by Outputs Within Project Foci 
(the function of an output) 



Output Dimension: Structure 


Output Dimension: 
Function 


Output 


Set DE 
Coding 


Cluster Categories: Products 


Policy 


Mgtnt 


Prod 


Totals 


01 


Reports/ Contracts 


0 


79 


43 


122 


04 


Work Specif i cat ions/ Procedures 


3 


38 


37 


78 


l n 


Data (including printouts) 


0 


38 


37 


75 


02 


Data Instrucments/ Techniques 


0 


27 


47 


74 


08 


Plans /Designs /Theories /Mode Is 


7 


48 


18 


73 


05 


Work Management Systems 


0 


57 


5 


62 


03 


Guides/Manuals 


3 


24 


29 


56 


10 


Publications 


1 


15 


36 


52 


07 


Resource Lists 


0 


17 


27 


44 


14 


Instructnl Units/Use w/Staff 


0 


4 


35 


39 


12 


Instructnl Units/ In-School 


0 


0 


29 


29 


13 


Instructnl Units/Extra-School 


0 


0 


18 


18 


11 


Promotional Materials 


0 


4 


13 


17 


28 


Taxonomies 


0 


0 


5 


5 


29 


Recommendations 


0 


4 


0 


4 


36 


Curricular Objectives 


0 


0 


3 


3 


37 


Critiques 


0 


3 


0 


3 


06 


Data Management Systems 


0 


2 


0 


2 


09 


Equipment /Too Is /Facilities 


0 


1 


1 


2 


42 


Prediction Variables 


0 


0 


1 


1 


Totals 




14 


361 


384 


759 



Output Dimension: Structure 


Output Dimension: 
Function 


Output 


Set DE 
Coding 


Cluster Categories: Events 


Policy 


Mgmt 


Prod 


Totals 


32 


Recruiting/ Placing Staff 


0 


21 


0 


21 


33 


Collecting/Analyzing Data 


0 


9 


7 


16 


52 


Adapting Materials 


0 


0 


12 


12 


43 


Establishing Opns Parameters 


7 


4 


0 


11 


17 


Workshops/ Institutes 


0 


4 


6 


10 


40 


Acquiring Supplies/Facilities 


0 


8 


2 


10 


3] 


Field Trials /Experiments 


0 


6 


1 


7 


38 


Training 


0 


7 


0 


7 


51 


Acquiring Funds 


0 


6 


0 


6 






ERIC 



APPENDIX 10 



(Continued) 



Output Dimension: Structure 


Output Dimension: 
Function 


Output 


Set DE 
Coding 


Cluster Categories: Events 


Policy 


Mgmt 


Prod 


Totals 


48 


Maintaining Accountability 


0 


5 


0 


5 


18 


Presentations 


0 


2 


2 


4 


19 


Benchmark Decisions 


1 


2 


1 


4 


50 


Disseminating Inf o/ Outputs 


0 


4 


0 


4 


30 


Committee/Consultant Meetings 


0 


3 


0 


3 


15 


Staff Meeting 


0 


2 


0 


2 


16 


Conferences/ Seminars 


1 


1 


0 


2 


39 


Updating of Machine Systems 


0 


1 


1 


2 


47 


Coordinating Efforts 


0 


2 


0 


2 


49 


Establishing Field Sites 


0 


2 


0 


2 


Totals 




9 


89 


32 


130 


Output Dimension: Structure 


Output Dimension: J 

Function 


Output 


Set DE 
Coding 


Cluster Categories: Conditions 


Policy 


Mgmt 


Prod 


Totals 



21 


Working Environment /Atmosphere 


0 


19 


0 


19 


20 


Cooperative Relationships 


1 


12 


1 


14 


34 


Coordinated Efforts 


1 


8 


1 


10 


22 


Quality Assurance 


1 


6 


0 


7 


23 


Accountability 


0 


6 


0 


6 


25 


Resource Accessibility 


0 


4 


0 


4 


41 


Effective Communication 


0 


3 


1 


4 


44 


Provision for Services 


0 


1 


3 


4 


45 


Informed Public 


0 


2 


0 


2 


26 


Available Field Sites 


0 


1 


0 


1 


35 


Acceptance/Adoption of Outputs 


0 


1 


0 


1 


46 


Trained Staff 


0 


1 


0 


1 



Totals 


3 


64 


6 


73 


Grand Totals 


26 


514 


422 


962 



338 




* 



APPENDIX 11 



The Uses Made of Outputs 
(the character of 



Within Project Foci 
an output) 



Output Dimension: Structure 


Output Dimension: 
Character 


Output 


Set DE 


Cluster Categories: Products 


Know 


Tech 


Imp 


Info 


; Totals 


Coding 












i 


01 


Reports /Con tracts 


10 


18 


16 


78 


122 


04 


Work Specif ications/Procedures 


0 


74 


0 


4 


78 


27 


Data (including printouts) 


2 


3 


1 


69 


75 


02 


Data Instrument s/ Techniques 


0 


74 


0 


0 


74 


08 


Plans /Designs /Theories /Models 


3 


69 


1 


0 


73 


05 


Work Management Systems 


2 


47 


9 


4 


62 


03 


Guides/Manuals 


0 


55 


0 


1 


56 


10 


Publications 


0 


6 


36 


10 


52 


07 


Resource Lists 


0 


33 


1 


10 


44 


14 


Instructnl Units/Use w/Staff 


0 


39 


0 


0 


39 


12 


Instructnl Units/ In-School * 


0 


28 


1 


0 


29 


13 


Instructnl Units/Extra-School 


0 


15 


3 


0 


18 


11 


Promotional Materials 


0 


1 


16 


0 


17 


28 


Taxonomies 


0 


5 


0 


0 


5 


29 


Recommendations 


0 


0 


0 


4 


4 


36 


Curricular Objectives 


0 


3 


0 


0 


3 


37 


Critiques 


0 


0 


0 


3 


3 


06 


Data Management Systems 


0 


2 


0 


0 


2 


09 


Equipment /Tools/Facil it ies 


0 


1 


1 


0 


2 


42 


Prediction Variables 


0 


0 


0 


1 


1 


Totals 




17 


473 


85 


184 


759 




Output Dimension: Structure 


Output Dimension: j 










Character 




Output 


Set DE 


Cluster Categories: Events 


Know 


Tech 


Imp 


Info 


Totals 


Coding 










• 




32 


Recruiting/Placing Staff 


2 


8 


8 


3 


21 


33 


Collecting/Analyzing Data 


1 


8 


0 


7 


16 


52 


Adapting Materials 


0 


12 


0 


0 


12 


43 


Establishing Opns Parameters 


0 


4 


2 


5 


11 


17 


Workshops/ Institutes 


0 


8 


2 


0 


10 


40 


Acquiring Supples /Facilities 


1 


6 


2 


1 


10 


31 


Field Trials /Experiments 


0 


7 


0 


0 


7 


38 


Training 


0 


4 


2 


1 


7 


51 


Acquiring Funds 


0 


0 


6 


0 


6 








345 



APPENDIX 11 



(Continued) 


Output Dimension: Structure 


Output Dimension: 
Character 


Output 

Totals 


Set DE 
Coding 


Cluster Categories: Events 


Know Tech Imp Info 



48 Maintaining Accountability 

18 Presentations 

19 Benchmark Decisions 

50 Disseminating Info/Output 

30 Committee/Consultant Meetings 

15 Staff Meeting 



2 

0 

1 

0 

0 

1 



3 

0 

3 

0 

0 

0 



0 

4 

0 

4 

1 

0 



0 

0 

0 

0 

2 

1 



5 

4 

4 

4 

3 

2 



O 

ERIC 



16 


Conferences/ Seminars 


0 


2 


0 


0 


2 


39 


Updating of Machine Systems 


0 


2 


0 


0 


2 


47 


Coordinating Efforts 


0 


2 


0 


0 


2 


49 


Establishing Field Sites 


1 


1 


0 


0 


2 


Totals 




9 


70 


31 


20 


130 




Output Dimension: Structure 


Output Dimension: 










Character 




Output 


Set DE 1 


Cluster Categories: Conditions 


Know 


Tech 


Imp 


Info 


Totals 


Coding 














21 


Working Environment /Atmosphere 


5 


5 


6 


3 


19 


20 


Cooperative Relationships 


2 


6 


5 


1 


14 


34 


Coordinated Efforts 


1 


6 


2 


1 


10 


22 


Quality Assurance 


1 


5 


1 


0 


7 


23 


Accountability 


2 


2 


1 


1 


6 


25 


Resource Accesibility 


1 


3 


0 


0 


4 


41 


Effective Communication 


1 


1 


2 


0 


4 


44 


Provision for Services 


0 


4 


0 


0 


4 


45 


Informed Public 


0 


0 


2 


0 


2 


26 


Available Field Sites 


1 


0 


0 


0 


1 


35 


Acceptance/Adoption of Outputs 


0 


0 


1 


0 


1 


46 


Trained Staff 


0 


1 


0 


0 


1 


Totals 




14 


33 


20 


6 


73 


Grand Totals 


40 


576 


136 


210 


962 




340 
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APPENDIX 12 



The Positions Held by Outputs Within Project Foci 
(the level of an output) 



Output Dimension: Structure 


Output Dimension: 








Level 


Output 


Set DE 
Coding 


Cluster Categories: Products 


Focal Compon Facil 


Totals 



01 


Reports/ Contracts 


A1 


18 


63 


122 


0A 


Work Specif icat ions /Procedures 


1 


19 


58 


78 


27 


Data (including printouts) 


6 


17 


52 


75 


02 


Data Instruments /Techniques 


1 


26 


A 7 


7A 


07 


Resource Lists 


7 


9 


57 


73 


08 


Plans /Designs /Theories /Models 


7 


9 


57 


73 


05 


Work Management Systems 


5 


5 


52 


62 


03 


Guides/Manuals 


9 


17 


30 


56 


10 


Publications 


21 


13 


18 


52 


1A 


Instructnl Units/Use w/Staff 


1 


28 


10 


39 


12 


Instructnl Unit s/ In-School 


6 


21 


2 


29 


13 


Instructnl Unit s /Extra-School 


A 


10 


A 


18 


11 


Promotional Materials 


1 


10 


6 


17 


28 


Taxonomies 


2 


3 


0 


5 


29 


Recommendations 


0 


0 


A 


A 


36 


Curricular Objectives 


0 


2 


1 


3 


37 


Critiques 


0 


0 


0 


3 


06 


Data Management Systems 


0 


0 


2 


2 


09 


Equipment /Tools /Facil it ies 


1 


0 


1 


2 


A2 


Prediction Variables 


0 


1 


0 


1 



Totals 119 212 A28 759 



Output Dimension: Structure 


Output Dimension: 
Level 


Output 


Set DE 
Coding 


Cluster Categories: Events 


Focal 


Compon 


Facil 


Totals 


32 


Recruiting/Placing Staff 


0 


0 


21 


21 


33 


Collecting /Analyzing Data 


0 


0 


16 


16 


52 


Adapting Materials 


0 


0 


12 


12 


A3 


Establishing Opns Parameters 


3 


0 


8 


11 


17 


Workshops/ Institutes 


0 


A 


6 


10 


A0 


Acquiring Supplies /Facil it ies 


0 


0 


10 


10 


31 


Field Trials/Experiments 


0 


0 


7 


7 


38 


Training 


1 


0 


6 


7 


51 


Acquiring Funds 


0 


0 


6 


6 



ERIC 



347 



APPENDIX 12 



(Continued) 



Output* Dimension: Structure 


Output Dimension: 
Level 


Output 


Set DE 


Cluster Categories: Events 


Focal 


Compon 


Facil 


Totals 


Coding 












48 


Maintaining Accountability 


0 


0 


5 


5 


18 


Presentations 


1 


1 


2 


4 


19 


Benchmark Decisions 


0 


0 


4 


4 


50 


Disseminating Info/ Outputs 


2 


0 


2 


4 


30 


Committee/Consultant Meetings 


0 


0 


3 


3 


15 


Staff Meeting 


0 


0 


2 


2 


16 


Conferences/ Seminars 


0 


0 


2 


2 


39 


Updating of Machine Systems 


0 


0 


2 


2 


47 


Coordinating Efforts 


0 


0 


2 


2 


49 


Establishing Field Sites 


0 


0 


2 


2 


Totals 




7 


5 


118 


130 


Output Dimension: Structure 


Output Dimension: 










Level 




Output 


Set DE 1 


Cluster Categories: Conditions 


Focal 


Compon 


Facil 


Totals 


Coding | 












21 


Working Env i r onmen t / Atmo sphere 


0 


0 


19 


19 


20 


Cooperative Relationships 


2 


0 


12 


14 


34 


Coordinated Efforts 


1 


0 


9 


10 


22 


Quality Assurance 


0 


0 


7 


7 


23 


Accountability 


0 


0 


6 


6 


25 


Resource Accessibility 


1 


0 


3 


4 


41 


Effect ive Communication 


0 


0 


4 


4 


44 


Provision for Services 


3 


0 


1 


4 


45 


Informed Public 


1 


0 


1 


2 


26 


Available Field Sites 


0 


0 


1 


1 


35 


Acceptance/Adoption of Outputs 


0 


0 


1 


1 


46 


Trained Staff 


0 


0 


1 


1 



Totals 


8 


0 


65 


73 


Grand Totals 


134 


217 


611 


962 



O 

ERLC 
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APPENDIX 13 



A Summary of the Standards Held for the Outputs Classified 
Within Frequently Used Output Cluster Categories 



Coding Sets 






DE 


Cluster Output Category 




j 


Structure of Standards 


Freq . 


LM 


Primary Category of Standard 




01 


Reports /Contracts 


26 


1 


Output Standards 


83 


01 


Completeness of content 


9 


02 


Quantity of outputs/data 


1 


04 


Communication and clarity 


7 


05 


Utility or value 


3 


06 


Acceptance by users 


1 


07 


Personal satisfaction /feeling 


3 


08 


Agreement/concurrence w/others 


5 


09 


Lack of errors/discrepancies 


5 


11 


Appropriate design/content 


3 


12 


Goal attainment 


10 


13 


Acceptance by others (in project) 


9 


14 


Acceptance by sponsor 


7 


15 


Compliance w/sponsor guideline 


3 


16 


Compares favorably 


4 


17 


Internally consistent 


2 


18 


Satisfactory appearance 


3 


19 


Logical criteria 


2 


21 


Sources of variance controlled 


2 


24 


Terminology appropriate 


3 


29 


Meets design expectations 


1 


2 


Process Standards 


16 


02 


Personnel are satisfied 


1 


04 


Deadlines are met 


4 


11 


Costs consistent w/estimates 


1 


17 


External enthusiasm evident 


1 


20 


Performance respected 


2 


24 


Cos ts acceptable for benefits 


1 


25 


Staff reflect trust 


1 


34 


Impact of effort favorable 


2 


35 


Outputs distributed/requested 


1 


39 


Evidence of pre-planning 


1 


40 


Outputs published externally 


1 
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APPENDIX 13 



(Continued) 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




J 


Structure of Standards 


Freq . 


LM 


Primary Category of Standard 




02 


Data Instruments /Techniques 


20 


1 


Output Standards 


64 


01 


Completeness of content 


5 


03 


Quantity of effort expended 


1 


04 


Communication and clarity 


3 


05 


Utility or value 


5 


07 


Personal satisfaction/feeling 


4 


08 


Agreement /concurrence w/others 


1 


09 


Lack of errors/discrepancies 


1 


11 


Appropriate design/content 


4 


12 


Goal attainment 


4 


13 


Acceptance by others (in project) 


10 


14 


Acceptance by sponsor 


1 


16 


Compares favorably 


1 


17 


Internally consistent 


1 


18 


Satisfactory appearance 


1 


19 


Logical criteria 


2 


20 


Performs consistently 


1 


21 


Sources of variance controlled 


2 


22 


Functions as planned 


10 


26 


Components are complementary 


1 


28 


Operable by others 


2 


30 


Lack of negative feedback 


2 


31 


Meets legal constraints 


1 


32 


Created by ruputable producer 


1 


2 


Process Standards 


4 


12 


No obvious omissions 


1 


17 


External enthusiasm evident 


2 


24 


Costs acceptable for benefits 


1 


04 


Work Specifications/Procedures 


21 


1 


Output Standards 


70 


01 


Completeness of content 


3 



APPENDIX 13 



(Continued) 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




J 


Structure of Standards 


Freq . 


LM 


Primary Category of Standard 




02 


Quantity of outputs/data 


i 


03 


Quantity of effort expended 


i 


04 


Communication and clarity 


9 


05 


Utility or value 


4 


06 


Acceptance by users 


1 


07 


Personal sat is fact ion/ feeling 


5 


08 


Agreement/concurrence w/others 


2 


09 


Lack of errors/discrepancies 


5 


11 


Appropriate design/content 


7 


12 


Goal attainment 


5 


13 


Acceptance by others (in project) 


6 


14 


Acceptance by sponsor 


3 


15 


Compliance w/sponsor guideline 


1 


17 


Internally consistent 


5 


18 


Satisfactory appearance 


3 


19 


Logical criteria 


4 


22 


Functions as planned 


3 


25 


Awareness that outputs exist 


2 


2 


Process Standards 


12 


03 


Minimum correction required 


1 


04 


Deadlines are met 


2 


07 


An expected activity occurs 


1 


14 


No felt deficiencies 


1 


17 


External enthusiasm evident 


4 


24 


Costs acceptable for benefits 


1 


28 


Closure reached on questions 


1 


34 


Impact of effort favorable 


1 


08 


Plans /Designs /Theories /Mode Is 


26 


1 


Output Standards 


82 


01 


Completeness of content 


11 


04 


Communication and clarity 


1 


05 


Utility or value 


7 


06 


Acceptance by users 


3 


07 


Personal satisfaction/ feeling 


3 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




J 


Structure of Standards 


Freq . 


LM 


Primary Category of Standard 




08 


Agreement /concurrence w/others 


2 


09 


Lack of errors/discrepancies 


3 


11 


Appropriate design /content 


5 


12 


Goal attainment 


4 


13 


Acceptance by others (in project) 


10 


14 


Acceptance by sponsor 


1 


15 


Compliance w/sponsor guideline 


4 


16 


Compares favorably 


7 


17 


Internally consistent 


2 


19 


Logical criteria 


2 


20 


Performs consistently 


2 


21 


Sources of variance controlled 


4 


22 


Functions as planned 


2 


23 


Successfully constrains/guides 


3 


24 


Terminology appropriate 


2 


26 


Components are complementary 


2 


30 


Lack of negative feedback 


2 


2 


Process Standards 


20 


04 


Deadlines are met 


1 


08 


Staff contributions accepted 


1 


09 


Outside contributions accepted 


2 


15 


Tasks perceived and acted upon 


1 


16 


External cooperation gained 


1 


21 


Follow-on proposals are funded 


1 


23 


Project view accepted 


1 


24 


Costs acceptable for benefits 


2 


28 


Closure reached on questions 


2 


34 


Impact of effort favorable 


1 


36 


Employment criteria met 


6 


41 


Guidelines are followed 


1 


20 


Cooperative Relationships 


12 


1 


Output Standards 


8 


02 


Quantity of outputs/data 


1 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




J 


Structure of Standards 


Freq . 


LM 


Primary Category of Standard 




06 


Acceptance by users 


1 


07 


Personal sat is fact ion/ feeling 


2 


13 


Acceptance by others (in project) 


1 


22 


Functions as planned 


2 


25 


Awareness that outputs exist 


1 


2 


Process Standards 


33 


02 


Personnel are satisfied 


L 


04 


Deadlines are met 


2 


05 


Acceptable level of output 


1 


07 


An expected activity occurs 


1 


08 


Staff contributions accepted 


1 


13 


Work conducted w/ in budget 


1 


15 


Tasks perceived and acted upon 


1 


16 


External cooperation gained 


3 


17 


External enthusiasm evident 


4 


20 


Performance respected 


1 


21 


Follow-on proposals are funded 


1 


22 


Feedback occurs 


1 


23 


Project view accepted 


1 


24 


Costs acceptable for benefits 


1 


25 


Staff reflect trust 


2 


26 


Values and objectives match 


2 


27 


Decisions result in action 


1 


29 


Creativity in work evidenced 


1 


34 


Impact of effort favorable 


6 


35 


Outputs distributed/requested 


1 


21 


Working Environment /Atmosphere 


15 


1 


Output Standards 


2 


04 


Communication and clarity 


1 


23 


Successfully constrains/guides 


1 


2 


Process Standards 


52 


01 


Personnel cooperate 


2 


02 


Personnel are satisfied 


5 


04 


Deadlines are met 


1 


05 


Acceptable level of output 


4 


07 


An expected activity occurs 


1 
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Coding Sets ! Output and Work Requirement Categories 

DE j Cluster Output Category 

J Structure of Standards 

EM I Priaary Category of Standard 



Freq 



10 


Maximum possible participation 


i 


12 


No obvious omissions 


i 


13 


Work conducted w/in budget 


2 


15 


Tasks perceived and acted upon 


11 


18 


Desired personnel obtained 


3 


20 


Performance respected 


3 


25 


Staff reflect trust 


4 


26 


Values and objectives match 


3 


29 


Creativity in work evidenced 


1 


30 


Personnel loss not excessive 


2 


33 


Personal growth/productivity 


1 


34 


Impact of effort favorable 


3 


43 


Policy constraints minimal 


1 


44 


Responsibility taken willingly 


1 


45 


Resources used efficiently 


1 


46 


Staff are self-direc t ing 


1 




Data (including printouts) 


13 




Output Standards 


46 


01 


Completeness of content 


4 


04 


Communication and clarity 


3 


05 


Utility or value 


7 


06 


Acceptance by users 


3 


07 


Personal sat isf act ion /feel ing 


4 


08 


Agree ment/concurrence w/others 


1 


09 


Lack of errors/discrepancies 


2 


11 


Appropriate design/content 


1 


12 


Goal attainment 


5 


13 


Acceptance by others (in project) 


4 


16 


Compares favorably 


3 


17 


IntP'.nally consistent 


1 


18 


Satisfactory appearance 


1 


19 


Logical criteria 


1 


21 


Sources of variance controlled 


3 


22 


Functions as planned 


2 


28 


Operable by others 


1 
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Coding Sets 


Output and Work Requirement Categories 


DE 


Cluster Output Category 




j 


Structure of Standards 


: Freci . 


LM 


Primary Category of Standard 


1 


2 


Process Standards 


14 


04 


Deadlines are met 


1 


05 


Acceptable level of output 


1 


07 


An expected activity occurs 


2 


11 


Costs consistent w/estimates 


2 


12 


No obvious omissions 


L 


14 


No felt deficiencies 


3 


24 


Costs acceptable for benefits 


2 


32 


Resources available on request 


1 


36 


Employment criteria met 


1 


32 


Recruiting/Placing Staff 


14 


1 


Output Standards 


14 


06 


Acceptance by users 


8 


07 


Personal satisfaction/ feeling 


2 


08 


Agreement /concurrence w/others 


2 


09 


Lack of errors/discrepancies 


1 


12 


Goal attainment 


1 


2 


Process Standards 


47 


01 


Personnel cooperate 


5 


03 


Minimum correction required 


1 


04 


Deadlines are met 


2 


05 


Acceptable level of output 


3 


06 


Work structure is efficient 


1 


07 


An expected activity occurs 


1 


12 


No obvious omissions 


1 


13 


Work conducted w/ in budget 


6 


14 


No felt deficiencies 


3 


15 


Tasks perceived and acted upon 


1 


17 


External enthusiasm evident 


1 


18 


Desired personnel obtained 


2 


19 


Adequate reputation w/sponsor 


1 


20 


Performance respected 


4 


26 


Values and objectives match 


2 


36 


Employment criteria met 


12 


37 


Staff adaptive to situations 


1 
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Coding SeL^ 


Output and Work Requirement Categories 




DE 


Cluster Output Category | 


: 


J 


Structure of Standards 


Freq . 


LM 


Primary Category of Standard 




33 


Collecting/Analyzing Data 


4 


1 


Output Standards 


14 


01 


Completeness of content 


1 


02 


Quantity of outputs/data 


1 


05 


Utility or value 


2 


07 


Personal satisfact ion/feeling 


2 


11 


Appropriate design/content 


1 


12 


Goal attainment 


2 


13 


Acceptance by others (in project) 


1 


16 


Compares favorably 


1 


21 


Sources of variance controlled 


1 


24 


Terminology appropriate 


1 


30 


Lack of negative feedback 


1 


2 


Process Standards 


4 


03 


Minimum correction required 


1 


07 


An expected activity occurs 


1 


27 


Decisions result in action 


1 


34 


Impact of effort favorable 


1 
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A Summary of the Tasks Pursued in Producing the Outputs 
Classified Within Frequently Used 
Output Cluster Categories 



Coding Sets 


Output and Work Requirement Categories 






Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Prlaary Category of Task 




01 


Available Field Sites 


26 


01 


Clarifying Problem Addressed 


22 


02 


Specify context /scope/ limits of effort 


1 


03 


Specify variables to be studied 


2 


06 


Observe relevant /related field opns 


1 


07 


Identify characteristics tgt population 


2 


12 


Review of /familiarization w/Materials 


16 


02 


Formulating objectives 


10 


02 


Conceptualize the product and context 


1 


03 


Formulate specific quest ion ^/hypotheses 


2 


0A 


Determine nature of rnsmt obje:tives 


1 


07 


Confer with colleagues/others re objects 


6 


^3 


Designing the output 


24 


03 


Determine relation of var iables/compnt s 


2 


05 


Specify/select grouping/ sampling 


1 


06 


Specify treatments/procedures/ strategies 


3 


07 


Conceptualize output application 


1 


08 


Translate objectives/data into meaning 


1 


09 


Visualize /specify format / appearance 


6 


10 


Specify/identify character of materials 


7 


17 


Specify measures to be used 


1 


19 


Specify kind of data analysis to be used 


1 


20 


State overall design 


1 


04 


Producing the output 


73 


01 


Con struct /assemble measurement tools 


7 


02 


Construct /assemble components of output 


24 


04 


Secure/select required materials/comps 


3 


05 


Produce output 


23 


08 


Select /produce data pro cessing /hand ling 


2 


09 


Translate outcomes/content to user terms 


2 


12 


Invent new production method/ Inst rument 


3 


13 


Construct /employ product ion tool s /aid s 


8 
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Coding Set" 1 



DE 



NO 



QR 



Output and Work Requirement Categories 

Cluster Output Category 

Task Cluster Category 

Primary Category of Task 



Freq . 



18 



05 



02 

03 

Oo 

07 

J9 

10 

11 

19 



06 



01 

03 

08 

10 

11 

14 

15 



07 



17 



21 



01 

03 

10 

13 

14 



22 



01 

04 

05 

06 
09 



Provide materials to producers 1 

Collecting/processing data 50 

Administer/ Implemen data instruments 21 

Record/score/classify/code raw data 10 

Perform analyses as specified/indicated 3 

Compile computational data 1 

Interpret data 12 

Acquire artifacts 1 

Try out output/component for operability 1 

Specif y /identify data identity codes 1 

Assessing the output quality 37 

Check output against specifications 5 

Determine achievement of objectives 1 

Determine inadequate output components 3 

Confirm operability/validity of tools 4 

Judge effectiveness of data gathering 1 

Note/make advisable modification(s) 7 

Proofread/edit written material 16 

Diffusing the output 3 

Distribute output/component 3 

Procuring professional staff 9 

Analyze work for competencies required 2 

Formulate job descriptions/qualif /salary 2 

Select/Hire/Appoint personnel 3 

Determine manpower needs 1 

Determine tasks to procure staff 1 

Effecting accountability 40 

Specify accountability procedures 2 

Establish/Maintain time lines 6 

Project cost estimates 2 

Determine nature/content of reporting 1 

Monitor expenditure of t ime/svcs /money 2 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




WO 


Task Cluster Category 


Freq . 


QR. 


Primary Category of Task 




10 


Report status as required/appropriate 


7 


n 


Explicate obj /outputs to be produced 


1 


12 


Explicate procedures/ st rategies/dut ies 


2 


14 


Issue production orders 


4 


16 


Compare expenditure rate with plan 


1 


19 


Maintain materials/use/f iscal records 


4 


21 


Develop record keeping system 


1 


22 


Est imate/Provide material/personnel 


1 


23 


Direct/Reques t special reports 


1 


27 


Monitor /Determine cost -effect iveness 


1 


28 


Specif y/ Establish/ Adopt policies 


2 


30 


Obtain staff input regarding time/funds 


1 


31 


Participate in production efforts as req 


1 


23 


Procuring systems/ services 


9 


01 


Determine tasks/services to be performed 


1 


02 


Determine f acilit ies/f ield material req 


1 


05 


Negotiate contract/coop w/sit es/services 


5 


07 


Arrange visitation/schedule as needed 


2 


24 


Effecting quality control 


33 


06 


Monitor adherence to output specs 


3 


09 


Initiate/Direct quality control action 


6 


10 


Monitor progress/status of work 


3 


11 


Proof /crit ique products /performances 


6 


14 


Determine need for/present alternatives 


1 


16 


Negotiate quality issues with sponsor 


4 


17 


Direct /request critiques from others 


6 


23 


Confirm accuracy of data inputs/outputs 


4 


25 


Maintaining job satisfaction 


2 


01 


Specify/ explain agency /project position 


1 


03 


Recognize merit 


1 


26 


Facilitating growth of staff 


2 



Of) Specify ownership/authorship policies 1 

07 Encourage publication of important work I 
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(Continued) 



Coding 

-W — 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


-Q R - , 


Primary Category of Task 





29 



Facilitating relationships 



17 



04 

14 

15 

16 
17 



30 



03 



31 



03 

08 

09 

12 

13 

15 



32 



04 

11 



02 



01 



01 

02 

03 

08 

11 

12 

14 



02 



01 

02 



Interact w/setting to elicit cooperation 
Interact with setting to collaborate 
Interact w/setting to enhance 
Make materials/experiences available 
Provide foreign language translations 

Effecting information flow patterns 

Insure that key decision points get info 

Diffusing information within project 

Produce/ circulate bullet ins /memos /news 
Communicate production guidelines 
Keep involved personnel informed generally 
Refer problem to appropriate specialists 
Record events (e.g., minutes, processes) 

Advise technically within expertise 

Diffusing information beyond project 

Report project at agency/other meetings 
Approve articles/papers for publication 

Data Instruments/Techniques 

Clarifying Problem Addressed 

Review Problem-relevant literature/ info 
Specify context/scope/limits of effort 
Specify variables to be studied 
Determine problems to be addressed 
Visualize application of technique/ ideas 

Review of /Familiarization w/materials 
Compare existing frameworks/procedures 

Formulating objectives 

Conceptualize the theoretical framework 
Conceptualize the product and context 



5 
2 
8 

1 

1 

4 

4 

13 

1 

7 

1 

2 

1 

1 

3 

2 

1 

20 

26 

6 

1 

3 

1 

3 

11 

1 

18 

2 

2 
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Coding Sets 


Output and Work Requirement Categories 




DE 

NO 

QR 


Cluster Output Category 

Task Cluster Category 

Primary Category of Task 


Freq . 


03 


Formulate specif ic quest ions /hypo theses 


3 


06 


Create objectives 


1 


07 


Confer with colleagues/others re objects 


10 


03 


Designing the output 


21 


0] 


Identify decision situations of interest 


1 


03 


Determine relation of var iables/compnts 


3 


06 


Specify treatments /procedures /strategies 


3 


07 


Conceptualize output application 


2 


09 


Visualize/specify format /appearance 


4 


10 


Specify/identify character of materials 


6 


12 


Es tab tolerances /effectiveness criteria 


1 


18 


Specify data/info processing/up-date 


1 


04 


Producing the output 


60 


02 


Construct /assemble components of output 


38 


05 


Produce output 


6 


07 


Package output 


3 


13 


Construct/employ production tools/aids 


1 


14 


Adapt materials for local/specific use 


9 


18 


Provide materials to producers 


1 


19 


Adapt content into processing format 


2 


05 


Collecting/processing data 


34 


01 


Conduct/operate program to be measured 


2 


02 


Administer/ Implement data instruments 


3 


03 


Record /score/ classify /code raw data 


6 


04 


Transfer raw data into storage form 


1 


05 


Translate data into processing form 


3 


06 


Perform analyses as specif ied/ indicated 


2 


08 


Prepare supporting documentation 


2 


09 


Interpret data 


7 


11 


Try out output /component for operability 


8 


06 


Assessing the output quality 


30 


01 


Check output against specifications 


3 


02 


Judge significance of performance result 


3 


06 


Judge output operability/acceptability 


3 
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Coding Sets 


Output and Work Requirement Categories 




DK 


Cluster Output Category 




NO 


Task Cluster Category 


Freq • 


QR 


Primary Category of Task 




o; 


Judge output effectiveness/consistency 


i 


03 


Determine inadequate output components 


2 


10 


Confirm operability/validity of tools 


1 


14 


Note/make advisable modification (s) 


15 


15 


Proofread/edit written material 


1 


20 


Select best alternative method/procedure 


1 


22 


Effecting accountability 


8 


02 


Delegate responsibilit ies /author it y 


3 


14 


Issue production orders 


2 


18 


Insure optimal use of resources 


1 


19 


Maintain materials /use/ fiscal records 


1 


26 


Divide work into appropriate segments 


1 


23 


Procuring systems /services 


7 


05 


Negotiate contract /coop w/sites/ services 


1 


06 


Get permission to use copyright material 


1 


07 


Arrange visitation/schedule as needed 


4 


08 


Make public service/material/ sit e needs 


1 


24 


Effecting quality control 


12 


07 


Monitor personnel performance 


1 


11 


Proof /critique products /performances 


6 


17 


Direct /request critiques from others 


3 


23 


Confirm accuracy of data inputs/outputs 


2 


29 


Facilitating relationships 


5 


04 


Interact w/setting to elicit cooperation 


1 


05 


Communicate behavior required of setting 


1 


14 


Interact with setting to collaborate 


3 


30 


Effecting information flow patterns 


4 


02 


Determine/use information sources 


3 


03 


Insure that key decision points get info 


1 


31 


Diffusing information within project 


17 


03 


Produce/ circulate bullet ins /memos / news 


2 


04 


Lead/participate in task-oriented groups 


5 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




05 


Join in/lead infor meetings/contacts 


2 


06 


Receive and react to relevant input 


3 


08 


Communicate production guidelines 


1 


09 


Keep involved personnel informed generally 


1 


15 


Advise technically within expertise 


3 


33 


Effecting decision mechanisms 


6 


03 


Make decisions 


6 


04 


Work Specif icat ions /Procedures 


21 


01 


Clarifying Problem Addressed 


31 


01 


Review Problem-relevant literature/info 


6 


02 


Specify context/scope/limits of effort 


4 


03 


Specify variables to be studied 


1 


06 


Observe relevant /related field opns 


2 


08 


Determine problems to be addressed 


1 


09 


Determine sensitive areas in tgt context 


1 


10 


Determine priorities of problems 


1 


12 


Review of /Familiarization w/materials 


9 


13 


Obtain literature relevant to problem 


1 


15 


Opera t ionally define terms/ variables 


5 


02 


Formulating objectives 


14 


01 


Conceptualize the theoretical framework 


1 


02 


Conceptualize the product and context 


2 


04 


Determine nature of msmt objectives 


1 


06 


Create obj ecjectives 


4 


07 


Confer with colleagues/others re objects 


6 


03 


Designing the output 


42 


02 


Isolate contaminating variables 


1 


03 


Determine relation of variables/compnt s 


6 


04 


Select /specif y populat ion (s) involved 


4 


05 


Specify/select grouping/ sampling 


2 


06 


Specify treatments/procedures/ strategies 


5 



3fc>2» 



357 



APPENDIX 14 



(Continued) 



Coding Setsj 


Output and Work Requirement Categories 




DE [ 


Cluster Output Category 




NO 1 


Task Cluster Category 


Freq . 


QR ! 


Primary Category of Task 




07 


Conceptualize output application 


1 


08 


Translate objectives/data into meaning 


1 


09 


Visualize /specify format /appearance 


3 


10 


Specify/identify character of materials 


7 


12 


Estab tolerance s/ef feet iveness criteria 


4 


.18 


Specify data/info processing/up-date 


1 


20 


State overall design 


3 


23 


Consider /design alternative approaches 


3 


24 


Visualize/Specify effect of output 


1 


04 


Producing the output 


23 


02 


Construct/assemble components of output 


7 


05 


Produce output 


7 


07 


Package output 


1 


08 


Select/produce data processing/handling 


3 


13 


Construe t /employ production tools/ a ids 


4 


14 


Adapt materials for local/specific use 


1 


05 


Collect ing/processing data 


36 


01 


Conduct /operate program to be measured 


2 


02 


Admin is ter /Implement data instruments 


5 


03 


Record/ score/classify /code raw data 


4 


04 


Transfer raw data into storage form 


1 


05 


Translate data into processing form 


4 


06 


Perform analyses as specif ied/indicated 


11 


07 


Compile computational data 


2 


08 


Prepare supporting documentation 


3 


09 


Interpret data 


1 


11 


Try out output /component for operability 


3 


06 


Assessing the output quality 


38 


01 


Check output against specifications 


3 


06 


Judge output operabi 1 ity/accept ability 


1 


07 


Judge output effectiveness/ consistency 


1 


08 


Determine inadequate output components 


5 


11 


Judge effectiveness of data gathering 


1 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


2L. 


Primary Category of Task 






14 


Note /make advisable mod if ication (s ) 


18 


15 


Proofread/edit written material 


6 


16 


Determine adeq of implementation process 


1 


17 


Check recommended changes specs 


1 


18 


Judge output against personal standards 


1 


07 


Diffusing the output 


2 


14 


Negotiate user acceptance/use 


2 


22 


Effecting accountability 


12 


02 


Delegate responsibilit ies /authority 


5 


04 


Establish/Maintain time lines 


4 


05 


Project cost estimates 


1 


13 


Interact with advisory services 


1 


14 


Issue production orders 


1 


23 


Procuring systems /services 


3 


05 


Negotiate contract/coop w/sites/services 


2 


07 


Arrange visitation/schedule as needed 


1 


24 


Effecting quality control 


20 


01 


Specify/recoramend criteria for output 


2 


06 


Monitor adherence to output specs 


1 


07 


Monitor personnel performance 


2 


10 


Monitor progress/status of work 


4 


11 


Proof /critique products/ performances 


1 


14 


Determine need for/present alternatives 


1 


16 


Negotiate quality issuer with sponsor 


1 


17 


Direct /request critiques from others 


7 


18 


Monitor staff awareness of responsibility 


1 


25 


Maintaining job satisfaction 


2 


01 


Specify /explain agency/ project position 


2 


26 


Facilitating growth of staff 


3 


01 


Provide for guidance within project 


3 
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Coding Sets Output and Work Requirement Categories 



DE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 





29 



04 

06 

10 

14 

15 

16 



30 



03 



31 



06 

08 

09 

12 



33 



03 



08 



01 



01 

02 

03 

04 
06 

08 

09 

10 
11 
12 



Facilitating relationships 12 

Interact w/setting to elicit cooperation 4 

Explain/Prepare rationale for procedure to site 1 
Promote or maintain focus of effort 1 

Interact with setting to collaborate 1 

Interact w/setting to enhance 3 

Make materials/experiences available 2 

Effecting information flow patterns 1 

Insure that key decision points get info 1 

Diffusing information within project 8 

Recieve and react to relevant input 4 

Communicate production guidelines 1 

Keep involved personnel informed generally 1 

Refer problem to appropriate specialists 2 

Effecting decision mechanisms 1 

Make decisions 1 

Plans /Designs /Theories /Mode Is 26 

Clarifying problem addressed 39 

Review Problem-relevant literature/info 8 

Specify context/scope/limits of effort 3 

Specify variables to be studied 2 

Identify contaminating variables 2 

Observe relevant/related field opns 1 

Determine problems to be addressed 3 

Determine sensitive areas in tgt context 1 

Determine priorities of problems 1 

Visualize application of technique/ideas 3 

Review of /Familiarization w/materials 13 



Operationally define terms/ variables 
Identify/Assess divergent points of view 
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1 
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Coding Sets 


Output and Work Requirement Categories 




dE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




02 


Formulat ing objectives 


30 


02 


Conceptualize the product and context 


4 


03 


Formulate specific questions/hypotheses 


2 


05 


State problem in context of theory 


3 


06 


Create objectives 


2 


07 


Confer with colleagues/others re objects 


19 


03 


Designing the output 


70 


01 


Identify decision situations of interest 


3 


02 


Isolate contaminating variables 


2 


03 


Determine relation of variables/compnt s 


5 


04 


Select /specif y population (s) involved 


3 


05 


Specif y / select group ing /samp ling 


11 


06 


Specify treatment s /procedures/ strategies 


13 


07 


Conceptualize output application 


4 


09 


Visualize /specify format /appearance 


1 


10 


Specif y/identify character of materials 


11 


12 


Estab tolerances/effectiveness criteria 


4 


13 


Establish specs for output try-out/demo 


1 


15 


Specify site adaptations required 


1 


17 


Specify measures to be used 


3 


18 


Specify data / info pro cess ing /up-date 


2 


19 


Specify kind of data analysis to be used 


2 


20 


State overall design 


3 


22 


Specify data collection schedule 


1 


04 


Producing the output 


42 


01 


onst rue t/assemble measurement tools 


7 


02 


Construct /assemble components of output 


21 


04 


Secu -e/ select required mateirals/ comps 


2 


05 


Produce output 


6 


13 


Construct /employ production tools/aids 


2 


14 


Adapt materials for local/specific use 


2 


17 


Organize presentation components 


2 



O 
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Coding Sets i 


Output and Work Requirement Categories 




DE ' 


Cluster Output Category 




NO | 


Task Cluster Category 


Freq . 


QR | 


Primary Category of Task 




05 


Collecting/processing data 


20 


01 


Conduct /operate program to be measured 


1 


02 


Administer / Implement data instruments 


7 


03 


Record/score/classif y/code raw data 


5 


06 


Perform analyses as specified/indicated 


2 


09 


Interpret data 


2 


11 


Try out output /component for operability 


2 


14 


Specify parameters of sub-routines 


1 


06 


Assessing the output quality 


39 


01 


Check output against specifications 


1 


08 


Determine inadequate output components 


4 


10 


Confirm operabil ity/validity of tools 


1 


11 


Judge effectiveness of data gathering 


1 


12 


Relate data to decision situations 


1 


14 


Note/make advisable modification (s) 


25 


15 


Proofread/edit written material 


5 


16 


Determine adeq of implementation process 


1 


07 


Diffusing the output 


4 


14 


Negotiate user acceptance/use 


3 


17 


Distribute output/component 


1 


21 


Procuring professional staff 


2 


01 


Analyze work for competencies required 


1 


08 


Identify potential candidates 


1 


22 


Effecting accountability 


18 


02 


Delegate responsibilities /authority 


1 


04 


Establish/Maintain time lines 


3 


05 


Project cost estimates 


1 


09 


Monitor expenditure of time/svcs/money 


1 


11 


Explicate obj /outputs to be produced 


1 


12 


Explicate procedures /strategies /duties 


1 


13 


Interact with advisory services 


2 


14 


Issue production orders 


8 
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Coding Sets 


Output and Work Requirement Categories 


DE 

NO 

QR 


Cluster Output Category J 

Task Cluster Category j Froq . 

Primary Category of Task | 


23 


Procuring systens/services 16 


02 

03 

04 
03 


Determine f acilit ies/f ield material req 1 
Identify/study possible sites/services 6 
Select/obtain sites/facilit ies/servicc ■? 7 
Negotiate contract /coop w/sites/services 2 


24 


Effecting quality control 26 


06 

07 

09 

11 

14 


Monitor adherence to output specs 3 
Monitor personnel performance 1 
Initiate/Direct quality control action 3 
Proof /critique products/performances 4 
Determine need for/present alternatives 1 


17 

22 

24 


Direct/request critiques from others 11 
Select outputs for further review 2 
Monior/Judge project impact 1 


26 


Facilitating growth of staff 1 


06 


Specify ownership/authorship policies 1 


29 


Facilitating relationships 18 


04 

05 

06 
07 
10 


Interact w/setting to elicit cooperation 3 
Communicate behavior required of setting 5 
Explain/Prepare rationale for procedure to site 4 
Negotiate resolution of differences 1 
Promote or maintain focus of effort 1 


13 

16 


Maintain contacts with field 2 

Make materials/experiences available 2 


30 


Effecting information flow patterns 2 


03 

07 


Insure that key decision points get info 1 

Explicate/employ ways of enhancing flow 1 


31 


Diffusing information within project 23 


01 

02 


Plan/conduct staff meetings/conferences 2 

Make formal presentations 2 



36? 
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Coding Sets j 


Output and Work Requirement Categories 




DE ' 

NO 

QR i 


Cluster Output Category 

Task Cluster Category 

Primary Category of Task 


Freq . 



03 


Produce /circulate bullet ins /memos /news 


2 


04 


Lead/participate in task-oriented groups 


4 


06 


Receive and react to relevant input 


4 


08 


Communicate production guidelines 


4 


09 


Keep involved personnel informed generally 


1 


10 


Communicate relevant experience info 


2 


12 


Refer problem to appropriate specialists 


1 


15 


Advise technically within expertise 


1 


32 


Diffusing information beyond project 


1 


08 


Interact with others regarding promotion 


1 


33 


Effecting decision mechanisms 


4 


03 


Make decisions 


3 


10 


Establish organizational structure 


1 




Cooperative Relationships 


12 


01 


Clarifying problem addressed 


9 


01 


Review Problem-relevant literature/ info 


1 


02 


Specify context /scope/limits of effort 


1 


06 


Observe relevant /related field opns 


2 


07 


Identify characteristics tgt population 


2 


09 


Determine sensitive areas in tgt context 


1 


10 


Determine priorities of problems 


1 


12 


Review of /Familiar ization w/materials 


1 


02 


Formulating objectives 


6 


01 


Conceptualize the theoretical framework 


1 


02 


Conceptualize the product and context 


2 


06 


Create objectives 


1 


07 


Confer with colleagues/others re objects 


2 


04 


Producing the output 


10 


02 


Construct/assemble components of output 


10 
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Cojlnfi Sets t 



DE 



Output and v ork Rr q/diement Categories 



NO 



OH 



Cluster Output Category 

Task Cluster Category 
Primary Category of Task 



Ereq 



05 



02 

04 



06 



07 

14 



07 



15 



21 



08 

09 

11 



22 



02 

12 

13 



23 



04 

05 



24 



01 

04 

07 

09 

10 

11 

19 



Collecting/processing data 

Administer/ Implement data instruments 
Transfer raw data into storage form 

Assessing the output quality 

Judge output effectiveness/consistency 
Note/make advisable modification (s) 

Diffusing the output 

Provide for user training as required 

Procuring professional staff 

Identify potential candidates 
Interview/ screen candidates 
Provide training as appropriate 

Effecting accountability 

Delegate responsibilit ies/ authority 
Explicate procedures/ strategies /duties 
Interact with advisory services 

Procuring systems/ services 

Select /obtain sites/ facili ties /services 
Negotiate contract/coop w/sites/ services 

Effecting quality control 

Specify/ recommend criteria for output 
Specify /develop improvement procedures 
Monitor personnel performance 
Initiate/Direct quality control action 
Monitor progress/status of work 

Proof /critique products/ performances 
Approve potential staff for employment 



1 

3 

1 

1 

5 

2 

1 

2 



2 

2 

1 



1 

2 

10 

1 

1 

2 

1 

1 

3 

1 



Maintaining job satisfaction 
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Coding Sets 


Output and Work Requirement Categories 




DE 


C] liter Output Category 




NO 


Task Cluster Category 


Free . 


QR | 


Primary Category of Task 





01 

02 

03 
06 

09 

10 



26 



02 

05 




29 



01 

04 

05 

07 

08 



09 

10 
11 

13 

14 

15 

16 



30 



01 

02 

03 

04 
07 



Specify/explain agency/project position 1 

Provide means for handling staff views 2 

Recoglize merit 1 

Recommend promotions/ salary increases 1 

Encourage optimal self-direction 2 

Foster mutual respect 1 

Facilitating growth of staff 2 

Provide opportunities for growth 1 

Provide opportunities for exposure 1 

Enhancing physical environment 1 

Optimize accessibility to equip/services 1 

Facilitating relationships 39 

Monitor field setting views/concerns 2 

Interact w/setting to elicit cooperation 2 

Communicate behavior required of setting 1 

Negotiate resolution of differences 2 

Locate/document sources of disagreement 2 

Adapt to accommodate differences 2 

Promote or maintain focus of effort 2 

Translate language into understood terms 2 

Maintain contacts with field 2 

Interact with setting to collaborate 5 

Interact w/setting to enhance 11 

Make materials/experiences available 6 

Effecting information flow patterns ? 

Define project information flow policies 2 

Determine/use information sources 1 

Insure that key decision points get info 4 

Provide communications equip/capability 1 

Explicate/employ ways of enhancing flow 1 

/Diffusing information within project 15 



31 
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Coding Sets 
DE 

NO 

_QR 

02 

03 

04 

05 

06 

09 
11 
12 



32 



02 

04 

06 

L0 



33 



03 

04 

07 

08 
10 



21 



21 



01 

02 

03 

07 

09 

10 
12 
17 



22 



02 

04 



Output and Work Requirement Categories 
Cluster Output Category 

Task Cluster Category 

Primary Category of Task 



Make formal presentations 
Produce /circulate bullet ins /memos /news 
Lead/participate in task-oriented groups 
Joi't iri/lead info meetings/contacts 
Receive and react to relevant input 

Keep involved personnel informed generally 
Advise on admin concerns w/in expertise 
Refer problem to appropriate specialists 

Diffusing information beyond project 

Prep are /disseminate promotional info 
Report project at agency/other meetings 
Interact with dissemination services 
Insure appropriate visibility of project 

Effecting decision mechanisms 

Make decisions 

Specify who needs what kinds of info 
Participate in decision-making process 
Ident if y/specif y sources of problems 
Establish organizational structure 

Working Environment/Atmosphere 

Procuring professional staff 

Analyze wor or competencies required 
Determine ^ petencies available 
Formulate job de script ions/ qualif / salary 
Make vacancies known 
Interview/ screen candidates 

Select /Hire/ Appoint personnel 
Negotiate w/candidate to employ/part ic 
Determine/assess best staffing pattern 

Ef f ect in 0 accountability 

Delegate responsibilities /authority 
Establish/Maintain time lines 

373 



Freq 



1 

2 

3 

1 

1 

5 
1 
1 

10 

1 

6 
2 
1 



1 

1 

1 

1 

1 

15 

31 

5 
3 
3 
3 

6 

8 

2 

1 

15 

8 

2 



O 
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Coding Setsl 


Output and Work Requirement Categories 


1 


DE | 

NO 

QR | 


Cluster Output Category 

Task Cluster Category 

Primary Category of Task 


! Freq . 


11 


Explicate obj/outputs to be produced 


2 


12 


Explicate procedures/ strategies/duties 


2 


26 


Divide woik into appropriate segments 


1 


24 


Effecting quality control 


13 


07 


Monitor personnel performance 


2 


09 


Initiate/Direct quality control action 


4 


10 


Monitor progress/status of work 


1 


11 


Proof /critique products/per fo nuances 


1 


13 


Monitor decisions made 


1 


14 


Determine need for/present alternatives 


1 


17 


Direct/request critiques from others 


1 


18 


Monitor staff awareness of responsibility 


1 


21 


Adjust staff for operation compatibility 


1 


25 


Maintaining job satisfaction 


29 


01 


Specify/explain agency/project position 


4 


02 


Provide means for handling staff views 


6 


03 


Recognize merit 


3 


04 


Counsel with/be supportive of personnel 


3 


05 


Adjust tasks to fit interests/limits 


2 


06 


Recommend promotions/ salary increases 


1 


09 


Encourage optimal self-direction 


5 


11 


Support staff in decisions they make 


1 


12 


Foster feelings of security in job 


1 


13 


Provide for social activities 


2 


15 


Nurture staff feelings of importance 


1 


26 


Facilitating growth of staff 


9 


01 


Provide for guidance w:thin project 


6 


02 


Provide opportunities for growth 


1 


05 


Provide opportunities for exposure 


1 


07 


Encourage publication of important work 


1 


27 


Enhancing physical environment 


5 


01 


Provide optimal proximity of staff 


2 


03 


Provide for privacy 


1 


04 


Provide meeting/relaxation facility 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




28 


Maintaining equity among staff 


2 


01 


Set demands in keeping with competencies 


1 


03 


Insure equity in application of rules 


1 


29 


Facilitating relationships 


5 


09 


Adapt to accommodate differences 


1 


14 


Interact with setting to collaborate 


2 


15 


Interact w/setting to enhance 


2 


30 


Effecting information flow patterns 


4 


01 


Define project information flow policies 


3 


05 


Define lines of communications 


1 


31 


Diffusing information within project 


6 


01 


Plan/ conduct staff meet ings/conf erences 


3 


04 


Lead/participate in task-oriented groups 


1 


09 


Keep involved personnel informed generally 


2 


33 


Effecting decision mechanisms 


9 


01 


Specif y/deterraine key decision points 


1 


02 


Specify kinds of decisions at each level 


3 


03 


Make decisions 


3 


06 


Specify vehicles for decision-making 




27 


Data (including printouts) 


13 


01 


Clarifying problem addressed 


8 


03 


Specify variables to be studied 


2 


10 


Determine priorities of problems 


2 


12 


Review of /Familiarization w/materials 


3 


13 


Obtain literature relevant to problem 


1 


02 


Formulating obj ect ives 


4 


02 


Conceptualize the product and context 


2 


05 


State problem in context of theory 


1 


07 


Confer with colleagues/others re objects 


1 
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Coding Sets' 


Output and Work Requirement Categories 





DF 


Cluster Output Category 


! 


NO 


Task Cluster Category 


Freq . 


Q R 


Primary Category of Task 




03 


Designing the output 


ii 


02 


Isolate contaminating variables 


2 


05 


Specify/ select grouping/ sampling 


2 


06 


Specify treatments/ procedures/ strategies 


1 


09 


Visualize /specify format /appearance 


1 


10 


Specify/identify character of materials 


2 


12 


Estab t ole ranee s/ef feet iveness criteria 


2 


23 


Consider/design alternatives approaches 


1 


04 


Producing the output 


20 


01 


Const rue t /assemble measurement tools 


2 


02 


Construct/assemble components of output 


9 


05 


Produce output 


2 


08 


Select/produce data processing/handling 


1 


12 


Invent new production method/instrument 


1 


13 


Construct /employ production tools/aids 


4 


16 


Coach talent (filming, video, sound) 


1 


05 


Collect ing /processing data 


60 


01 


Conduct/operate program to be measured 


1 


02 


Admin is ter/ Implement data instruments 


14 


03 


Record/ score /classify/ code raw data 


14 


04 


Transfer raw data into storage form 


5 


05 


Translate data into processing form 


4 


06 


Perform analyses as specified/indicated 


4 


07 


Compile computational data 


4 


08 


Prepare supporting documentation 


1 


09 


Interpret data 


11 


10 


Acquire artifacts 


1 


11 


Try out output/component for operability 


1 


06 


Assessing the output quality 


17 


01 


Check output against specifications 


1 


02 


Judge significance of performance result 


1 


05 


Identify unexpected /in congruent outcomes 


1 
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Coding Sets 


Output and Work Requirement Categories | 


DE 


Cluster Output Category 




NO 


Task Cluster Category 


1 I* req . 


QR 


Primary Category of Task 


1 


07 


Judge output effectiveness/consistency 


j 


08 


Determine inadequate output components 


4 


10 


Confirm operabil ity/validity of tools 


1 


11 


Judge effectiveness of data gathering 


1 


12 


Relate data to decision situations 


1 


14 


Note /make advisable modification (s) 


4 


15 


Proofread/edit written material 


2 


07 


Diffusing the output 


3 


15 


Provide for user training as required 


2 


17 


Distribute output /component 


1 


21 


Procuring professional staff 


17 


07 


Make vacancies known 


1 


09 


Interview/ screen candidates 


1 


10 


Select /Hire /Appoint personnel 


4 


11 


Provide training as appropriate 


10 


13 


Deter., le manpower needs 


1 


22 


Effecting accountability 


10 


04 


Establish/Maintain time lines 


1 


05 


Project cost estimates 


2 


12 


Explicate procedures/ strategies /duties 


1 


17 


Maintain duplicate records/files 


1 


18 


Insure optimal use of resources 


1 


19 


Maintain mate rials /use/ f is cal records 


3 


28 


Specify /Establish/ Adopt policies 


1 


23 


Procuring systems /services 


6 


02 


Determine f acilities/f ield material req 


1 


04 


Select/obtain sites/ facil it ies/services 


1 


07 


Arrange visitation/schedule as needed 


3 


09 


Procure required tools/equipment 


1 


24 


Effecting quality control 


9 


04 


Specify /develop improvement procedures 


1 
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Coding Setsf 


Output and Work Requirement Categories 


i 

Freq . 


DE 

NO 

OR , 


Cluster Output Category 

T\\sk Cluster Category 

Primary Category of Task 


06 


Monitor adherence to output specs 


3 


07 


Monitor personnel performance 


2 


08 


Monitor training effectiveness 


1 


10 


Monitor progress/status of work 


1 


17 


Direct/request critiques from others 


1 


?.9 


Facilitating relationships 


17 


04 


Interact w/setting to elicit cooperation 


3 


05 


Communicate behavior required of setting 


4 


06 


Explain/Prepare rationale for procedure to site 


1 


10 


Promote or maintain focus of effort 


2 


14 


Interact with setting to colloraborate 


3 


16 


Make materials /experiences available 


4 


30 


Effecting information flow patterns 


2 


03 


Insure that key decision points get info 


2 


31 


Diffusing information within project 


8 


01 


PI an /conduct staff meet ings /conferences 


1 


02 


Make formal presentations 


1 


03 


Produce /circulate bullet ins /memos /news 


2 


04 


Lead/participate in task-oriented groups 


1 


09 


Keep involved personnel informed generally 


3 


33 


Effecting decision mechanisms 


2 


03 


Make decisions 


2 


32 


Recruiting/Placing Staff 


14 


01 


Clarifying problem addressed 


5 


08 


Determine problems to be addressed 


1 


11 


Visualize application of technique/ideas 


4 


03 


Designing the output 


4 


08 


Translate objectives/data into meaning 


4 


21 


Procuring professional staff 


55 
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Coding Sets 


Output and Nork Requirement Categories 




flR 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




01 


Analyze work for competencies required 


3 


02 


Determine competencies available 


4 


03 


Formulate job descript ions/qualif /salary 


1 


06 


Maintain list of replacement personnel 


1 


07 


Make vacancies known 


4 


08 


Identify potential candidates 


9 


09 


Interview/screen candidates 


13 


10 


Select /Hire /Appoint personnel 


7 


12 


Negotiate w/candidate to employ/par t ic 


5 


13 


Determine manpower needs 


2 


15 


Recommend personnel for employment 


2 


16 


Obtain approval to hire 


1 


18 


Obtain/Assess candidate’s goals 


1 


19 


Review opns relative to existing staff 


1 


20 


Confer w/others re candidate selection 


1 


22 


Effecting accountability 


12 


02 


Delegate responsibilit ies /authority 


3 


03 


Determine/ verify /monitor budget balances 


2 


04 


Establish/Maintain time lines 


1 


11 


Explicate obj /outputs to be produced 


1 


12 


Explicate procedures /strategies/ duties 


5 


23 


Procuring systems/services 


27 


01 


Determine tasks/services to be performed 


2 


03 


Identify/study possible sites/services 


6 


04 


Select /obtain sites/ facil it ies /services 


1 


05 


Negotiate contract /coop w/sites/services 


17 


08 


Make public service/material/site needs 


1 


24 


Effecting quality control 


4 


05 


Delegate quality control responsibility 


1 


07 


Monitor personnel performance 


1 


10 


Monitor progress/status of work 


1 


11 


Proof /critique product s /per forraances 


1 


25 


Maintaining job satisfaction 


13 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


m 


Primary Category of Task 




01 


Specif y/explain agency/project position 


12 


05 


Adjust tasks to fit interest s/limits 


X 


25 


Facilitating growth of staff 


1 


01 


Provide for guidance within project 


1 


23 


Maintaining equity among stalf 


3 


01 


Set demands in keeping with competencies 


1 


02 


Determine excessive/low work loads 


2 


29 


Facilitating relationships 


2 


05 


Communicate behavior required of setting 


1 


13 


Maintain contacts with field 


1 


30 


Effecting information flow patterns 


1 


03 


Insure that key decision points get info 


1 


33 


Effecting decision mechanisms 


1 


03 


Make decisions 


1 


33 


Collecting/Analyzing Data 


4 


01 


Clarifying problem addressed 


2 


04 


Identify contaminating variables 


1 


10 


Determine priorities of problems 


1 


02 


Formulating objectives 


2 


07 


Confer with colleagues/others re objects 


2 


03 


Designing tb^ output 


2 


05 


Specify/select grouping/sampling 


1 


10 


Specify/identify character of materials 


1 


04 


Producing the output 


12 


02 


Construct/assemble components of output 


3 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


P imary Category of Task 





04 

08 

12 

13 



05 



02 

04 

07 



06 



01 

05 

07 

08 
14 



07 



17 



22 



04 

05 
17 



24 



07 

09 

11 

23 



25 



01 



29 



14 



Secure/select required materials/comps 
Seltct/produue data processing/handling 
Invent new production method/ instrument 
Construct/employ production tools/aids 

Collecting/processing data 

Administer/ Implement data instruments 
Transfer raw data into storage form 
Compile computational data 

Assessing the output quality 

Check output against specifications 
Identify unexpected/ incongruent outcomes 
Judge output effectiveness/consistency 
Determine inadequate output components 
Note/make advisable modification (s) 

Diffusing the output 

Distribute output /component 

Effecting accountability 

Establish/Maintain time lines 
Project cost estimates 
Maintain duplicate records/files 

Effecting quality control 

Monitor personnel performance 
Initiate/Direct quality control action 
Proof /critique pvoducts/performances 
Confirm accuracy of data inputs/outputs 

Maintaining job satisfaction 

Specify/explain agency/project position 

Facilitating relationships 

Interact with setting to collaborate 



1 

5 

1 

2 



1 

1 

2 

11 

5 

1 

1 

2 

2 

1 

1 

3 

1 

1 

1 



1 

1 

2 

1 

2 

2 

1 

1 
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Coding Seta' Output and Work Requirement Catego ries 

DE Cluster Output Category 

NO Task Cluster Category Freq 

QR Prim ar y Category of Te sk 



30 


Effecting information flow patterns 2 1 


03 


Insure that key decision points get info 2 \ 


31 


Diffusing information within project 4 I 


12 

15 


Refer problem to appropriate specialists 1 I 

Advise technically within expertise 3 J 


33 


Effecting decision mechanisms 1 | 


09 


Request support for decisions made 1 



0 
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APPENDIX 15 



A Summary of the Knowledges, Skills, and Sensitivities Used 
in Producirg the Outputs Classified with Frequently 
Used Output Cluster Categories 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




s 


Structure of Enabler 


Kreq . 


.. JY 


Primary Category of Enabler 





01 


Reports/Contracts 


26 


1 


Knowledge 


102 


01 


Standard school subjects 


3 


02 


Subjects learned in courses 


8 


03 


Subjects related to RDD&E 


21 


04 


Technical/ professional topics 


19 


05 


Project focus topics, external 


6 


06 


Project variables: external 


14 


07 


Project operation: general 


5 


08 


Project operation: specific 


6 


0? 


Scheduling and organizing 


3 


11 


Fiscal matters 


2 


12 


Resources: personnel 


2 


16 


Guidelines for reporting 


4 


17 


Writing styles 


1 


19 


Technical terminology /language 


2 


21 


Managemen t techniques 


2 


22 


Use of equipment /systems 


1 


23 


Char’s of target audience 


1 


24 


Process implementation (proj ) 


1 


30 


Errors in strategies/ j udgment 


1 


2 


Skill (or ability to perform) 


140 


01 


Teaching 


3 


02 


Facilitating people interactions 


11 


04 


Using/applying feedback 


4 


05 


Programming project events 


5 


06 


Programming subject matter 


1 


09 


Analytical problem solving 


3 


10 


Analytical data handling 


13 


11 


Disciplining self 


3 


13 


Listening 


3 


14 


Writing 


26 
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(Continued) 



Cod lug Sets I 

i)K ’ 

s 

;v 



Output and Work Requirement Categories 



Cluster Output Category 

Structure of Enabler 
Primary Category of Enable r 



Freq , 



15 

I 7 
18 

19 

20 

II 
22 

24 

25 
27 

29 

30 

31 

32 

34 

35 

44 

45 



01 

02 

03 

04 

05 

07 

09 

10 
13 

15 

16 

17 

18 

19 

20 



Presenting orally 
Interpreting language 
Finding fits integrating 
Planning/conceptual izing 
Exercising judgment 

Tracking activities/goals 
Estimating expenses/resources 
Explicating goals/procedures 
Applying measurement tools 
Using equipment/systems 

Getting others to perform 
Adapting to situation/demands 
Taking another's perspective 
Identifying/correcting errors 
Coordinating activities 

Communicating clearly 
Fiscal accounting 
Assessing personal performance 
Recalling 

Sensitivity (or awareness) 

Values of self and others 
Capabilities and limitations 
Needs of self and others 
Interactions of self and others 
Context of subject matter 

Context of objectives 
Awareness of alternatives 
Awareness of structure 
Language barriers 
Degress of freedom to deviate 

Existing value systems 
Personality of others 
Potential conflict of interest 
Supportiveness required 
Unstated obligations 



2 

4 

14 

4 

4 

2 

1 

2 

12 

7 

3 

3 

1 

1 

1 

3 
1 
2 
1 

73 

1 

4 

5 
7 

3 

1 

1 

2 

1 

4 

3 

3 

1 

1 

1 
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(Continued) 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




S 


Structure of Enabler 


Freq . 


uv 


Primary Category of Enabler 




21 


Limitations of analyses/data 


1 


22 


Responses of target audiences 


6 


23 


Cost /benefit factors 


2 


24 


Sources of error 


2 


25 


Individual differences 


3 


27 


Acceptability of output 


4 


30 


Response sets of target audiences 


3 


33 


Need for excellence in work 


2 


36 


Feeling of personal security 


1 


38 


Emerging directions 


1 


40 


Contractual /stated obi igations 


2 


41 


Inquisitiveness 


1 


42 


Enthusiasm 


2 


44 


Interested in activities 


1 


55 


Respect for/trust in others 


4 


02 


Data Inst ruments /Techniques 


20 


l 


Knowledge 


79 


01 


Standard school subjects 


1 


02 


Subjects learned in courses 


1 


03 


Subjects related to RDD&E 


22 


04 


Technical/ professional topics 


3 


05 


Project focus topics, external 


5 


06 


Project variables: external 


8 


07 


Project operation: general 


2 


08 


Project operation: specific 


16 


11 


Fiscal matters 


1 


12 


Resources: personnel 


2 


14 


Resources: time 


1 


17 


Writing styles 


1 


19 


Technical terminology /language 


1 


22 


Use of equipment/systems 


4 


23 


Char’s of target audience 


3 


24 


Process implementation (proj) 


6 


25 


Sources of info/materials 


1 


28 


Group dyn/decision processes 


1 



Sw 



385 



APPENDIX 15 





(Continued) 




Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




s 


Structure of Enabler 


Freq . 


uv 


Primary Category of Enabler 






Skill (or ability to perform) 


47 


01 


Teaching 


1 


02 


Facilitating people interactions 


7 


05 


Programming project events 


1 


Of) 


Programming subject matter 


1 


11 


Disciplining self 


2 


14 


Writing 


5 


17 


Interpreting language 


2 


18 


Finding fits integrating 


3 


19 


Planning /conceptual izing 


5 


21 


Tracking activities/goals 


2 


23 


Persuading /just ify ing 


1 


25 


Applying measurement tools 


1 


26 


Locating/maintaining info 


2 


28 


Running task oriented meetings 


1 


31 


Taking another 1 s perspective 


2 


34 


Coordinating activities 


1 


35 


Communicating clearly 


6 


38 


Using resources effectively 


1 


39 


Constructing measurement tools 


3 


3 


Sensitivity (or awareness) 


35 


02 


Capabilities and limitations 


4 


03 


Needs of self and others 


1 


04 


Interactions of self and others 


2 


06 


Worth in disciplines/methods 


2 


L3 


Language barriers 


1 


l- : 


Existing value systems 


2 


18 


Potential conflict of interest 


1 


21 


Limitations of analyses/data 


1 


22 


Responses of target audiences 


1 


23 


Cost/benefit factors 


1 


24 


Sources of error 


1 


30 


Response sets of target audiences 


3 


31 


Nature/scope of output 


2 


33 


Need for excellence in work 


2 


34 


Willingness to accept guidance 


1 
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(Cont inued) 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




S 


Structure of Enabler 


Freq . 


uv 


Primary Category of Enabler 




35 


Need to communicate effectively 


1 


36 


Feeling of personal security 


1 


38 


Emerging directions 


2 


41 


Inquisit iveness 


1 


42 


Enthusiasm 


1 


44 


Interest *d in activities 


2 


47 


Willingness to work as needed 


1 


56 


Habit of thoroughness 


1 


04 


Work Specif icat ions /Procedures 


21 


1 


Knowledge 


94 


01 


Standard school subjects 


3 


02 


Subjects learned in courses 


7 


03 


Subjects related to RDD&E 


12 


04 


Technical /professional topics 


18 


05 


Project focus topics, external 


5 


06 


Project variables: external 


10 


07 


Project operation: general 


6 


08 


Project operation: specific 


10 


09 


Scheduling and organizing 


6 


10 


Staff status/responsibilit ies 


1 


12 


Resources : personnel 


1 


17 


Writing styles 


1 


18 


Staff competencies /interests 


2 


21 


Management techniques 


2 


22 


Use of equipment/systems 


3 


23 


Char’s of target audience 


2 


24 


Process implementation (proj) 


2 


25 


Sources of info/materials 


2 


27 


Potential field settings 


1 


2 


Skill (or ability to perform) 


59 


01 


Teaching 


2 


02 


Facilitating people interactions 


8 


05 


Programming project events 


3 


06 


Programming subject matter 


3 


08 


Analyt ical reading/study 


2 
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(Continued) 



Cuding Sets 


Output and Work Requirement Categories 




UK 


Cluster Output Category 




s 


Structure of Enabler 


Freq . 


;:v 


Primary Category of Enabler 




u‘i 


Analytical problem solving 


2 


; o 


Analytical data handling 


2 


; i 


Disciplining Belf 


3 


14 


Writing 


7 


J 


Presenting orally 


1 


1/ 


Interpreting language 


2 


1 


Finding fits integrating 


12 


19 


Planning/conceptualizing 


3 


20 


Exercising judgment 


1 


24 


Explicating goals/procedures 


2 


27 


Using equipment/systems 
TaKing another’s perspective 


1 


J] 


2 


52 


Identifying/correcting errors 


1 


>5 


Communicating clearly 


1 


40 


Defining terms/ problems 


1 


5 


Sensitivity (or awareness) 


32 


UL 


Values of self and others 


2 


02 


Capabilities and limitations 


5 


03 


Context of subject matter 


2 


10 


Awareness of structure 


3 


14 


Reality in goal setting 


1 


16 


Existing value systems 


3 


17 


Personality of others 


1 


21 


Limitations of analyses/data 


1 


22 


Responses of target audiences 


3 


27 


Acceptability of output 


1 


30 


Response sets of target audiences 


3 


31 


Nature/scope of output 


3 


35 


Need to communicate effectively 


1 


42 


Enthusiasm 


1 


43 


Challenged by activities 


1 


44 


Interested in activities 


1 


08 


Plans /Designs /Theories /Mode Is 


26 


1 


Knowledge 


91 


01 


Standard school subjects 


2 



v 
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(Cont inued) 




Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




S 


Structure of Enabler 


Freq. 


IJV 


Primary Category of Enabler 





02 


Subjects learned in courses 


03 


Subjects related to RDD&E 


04 


Technical /professional topics 


05 


Project focus topics, external 


06 


Project variables: external 


07 


Project operation: general 


08 


Project operation: specific 


12 


Resources : personnel 


18 


Staff competencies/ interests 


22 


Use of equipment/systems 


23 


Char’s of target audience 


25 


Sources of info/materials 


27 


Potential field settings 
Skill (or ability to perform) 


01 


Teaching 


02 


Facilitating people interactions 


05 


Programming project events 


06 


Programming subject matter 


07 


Programming technical equip 


08 


Analytical reading/study 


09 


Analytical problem solving 


10 


Analytical data handling 


11 


Disciplining self 


13 


Listening 


14 


Writing 


17 


Interpreting language 


18 


Finding fits integrating 


19 


Planning/conceptualiz ing 


20 


Exercising judgment 


21 


Tracking activities/goals 


22 


Estimating expenses/resources 


23 


Persuading/ justifying 


24 


Explicating goals/procedures 


25 


Applying measurement tools 


27 


Using equipment/systems 


30 


Adapting to situation/demands 


33 


Graphically illustrating 


35 


Communicating clearly 

S' ■■ 

■■ SJ83 



1 

35 

13 

4 

If) 

3 

4 
3 

3 
2 

2 

1 

2 

68 

2 

8 

2 

1 

2 

1 

4 
4 
3 
1 



6 

1 

1 

1 

2 

1 

1 

4 

2 



2 

1 
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(Continued) 



Coding -Ip 1 8 


•"iifom nnd Work Requirement Categories 


Freq . 


')K 

rv 


"-.i.ito* Output Category 
Structure of Enabler 

Primary Category of Enabler 


7 


Vtvo*:3lng skills/potential 


i 


7 ' 1 


Constructing measurement tools 


i 




inivlty (or awareness) 


90 


1 


7 < lues of self and others 


6 


) ! 


Capabilities and limitations 


12 




Needs of self and others 


4 


) 4 


Interactions of self and others 


1 




Worth in disciplines/methods 


4 


) : 


Context of objectives 


2 


OS 


Worth in objectives 


1 


K* 


Awareness of alternatives 


3 


* 0 


Awareness of structure 


5 


i L 


Awareness of method 


3 


12 


Role of catalyst/synthesizer 


1 


A. J 


Language barriers 


1 




Reality in goal setting 


2 


In 


Existing value systems 


15 


11 


Personality of others 


2 


IS 


Potential conflict of interest 


2 


■i i 


Limitations of analyses/data 


3 


j 


Responses of target audiences 


2 


'> u 


Sources of eiror 


1 


26 


Recognition o£ data needs 


1 


27 


Acceptability of output 


1 


2.8 


Admitting error/adapting 


1 


.. 9 


Willingness to experiment 


2 


70 


Response sets of target audiences 


2 


31 


Nature/scope of output 


6 


34 


Willingness to accept guidance 


1 


IP 


Emerging directions 


1 


40 


Contractual/stated obligations 


1 


4 7 


Willingness to work as needed 


1 


48 


Common sense 


1 


34 


Manageability of data 


1 


57 


Deadlines 


1 
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(Continued) 



Coding Sets 


Output and Work Requirement Categories 


DF- 


Cluster Output Category 


s 


Structure of Enabler 


UV 


Primary Category of Enabler 



20 Cooperative Relationships ] 2 



1 Knowledge 22 



04 


Technical /professional topics 


3 


05 


Project focus topics, external 




06 


Project variables: external 


1 1 


08 


Project operation: specific 


1 


09 


Scheduling and organizing 


2 


10 


Staff status /responsibil it ies 


2 


23 


Char’s of target audience 


1 


2 


Skill (or ability to perform) 


25 


02 


Facilitating people interactions 


7 


03 


Translating content to media 


2 


05 


Programming project events 


1 


12 


Disciplining others 


1 


15 


Presenting orally 


2 


17 


Interpreting language 


1 


18 


Finding fits integrating 


1 


20 


Exercising judgment 


1 


21 


Tracking activities/goals 


1 


23 


Persuading/ justifying 


1 


24 


Explicating goals/procedures 


2 


29 


Getting others to perform 


2 


34 


Coordinating activities 


1 


37 


Assessing skills /potential 


2 


3 


Sensitivity (or awareness) 


25 


01 


Values of self and others 


3 


02 


Capabilities and limitations 


3 


03 


Neels of self and others 


2 


C4 


Interactions of self and others 


2 


10 


Awareness of structure 


1 


13 


Language barriers 


3 


16 


Existing value systems 


4 


17 


Personality of others 


2 


18 


Potential conflict of interest 


1 


19 


Supportiveness required 


1 
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(Continued) 



Coding Sets \ 


Output and Work Requirement Categories 




bi* 


ClusLci Output Oategoiy 




,s 


Sltuilute of Enabler 


Freq , 


. . 


Primary Category of Enabler 


) '4 


Wiillngneas to accept guidance 


1 


ib 


to communicate effectively 


1 


t 


Willingnpfis »o delegate 


1 




. . ivlronment/Atmosphere 


15 




edge 


23 




Project focus topics, external 


1 


l)h 


Project variables: external 


2 


") 


Project operation: general 


2 


f iV 


Scheduling and organizing 


1 


!.<) 


Staff status/responsibilities 


4 


1 2 


Resources: personnel 


6 


18 


Staff competencies/interests 


2 


> 1 


Management techniques 


1 


; 1 


Process implementation (proj) 


1 


'b 


Sources of info/materials 


1 


VS 


Croup dyn/decision processes 


2 




■ f-.i ’1 (or ability to perform) 


24 


»: j 


Teaching 


1 


•. : 


Facilitating people interactions 


2 


05 


Programming project events 


5 


04 


Analytical problem solving 


1 


n 


Disciplining self 


2 


13 


Listening 


1 


] b 


Finding fits integrating 


2 


14 


Planning /conceptualizing 


1 


a 


Tracking activities/goals 


1 


a 


Persuading/justifying 


1 


>4 


Explicating goals/procedures 


1 


29 


Getting others to perform 


1 


34 


Coordinating activities 


2 


37 


Assessing skills/potential 


2 


38 


Using resources effectively 


1 



Sensitivity (or awareness) 




Values of self and others 
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Coding Sets 


Output and Work Requirement Categories 


_ _ j 


DE 

S 

UV 


Cluster Output Category 

Structure of Enabler 

Primary Category of Enabler 


i 

1 Frcq . 
i 


02 


Capabilities and limitations 


2 


03 


Needs of self and others 


8 


08 


Worth In objectives 


1 


10 


Awareness of structure 


1 


16 


Existing value systems 


2 


17 


Personality of others 


3 


18 


Potential conilict of interest 


3 


19 


Supportiveness required 


2 


20 


Unstated obligations 


0 


22 


Responses of target audiences 


1 


25 


Individual differences 


1 


29 


Willingness to experiment 


1 


30 


Response sets of target audiences 


1 


35 


Need to communicate effectively 


1 


36 


Feeling of personal security 


1 


37 


Willingness to delegate 


2 


47 


Willingness to work as needed 


2 


55 


Respect for/trust in others 


1 


58 


Willingness to support staff 


1 


27 


Data (including printouts) 


13 


1 


Knowledge 


50 


02 


Subjects learned in courses 


2 


03 


Subjects related to RDD&E 


11 


04 


Technical /professional topics 


5 


05 


Project focus topics , general 


6 


06 


Project variables: external 


7 


07 


Project operation: general 


4 


08 


Project operation: specific 


5 


09 


Scheduling and organizing 


L 


12 


Resources : personnel 


1 


22 


Use of equipment /systems 


1 


23 


Char’s of target audience 


1 


24 


Process implementation (pro j ) 


6 



02 



Skill (or ability to perform) 

Facilitating people interactions 



44 

3 
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cut 'nurd) 



i»it nut and Work Requirement Categories 




_ „ > 


ji . nf-rv ' it pur ( itegory 
mature, r*' r,iabler 




Freq . 


i rimary Category of Enabler 







ire//»pplyir.g feedback 1 

i rrmnlrr, project events 1 

ercrrninr j'-cf matter 1 

/.t/lvticnl dote handling 2 

’•* *•' ipl inine •’elf 3 

’ -• it ing 3 

'F-Ing fits integrating 4 

• i annlng/conceptual lzlng 3 

exercising Judgment 4 

estimating expenses/rcsourccs 1 

i i-rsuadlng/Justifying 1 

Applying measurement tools 7 

i icatlng/malntaining info 1 

< tfing others to perform 1 

adapting to situation/demands 1 

* .entif ylng/correcting errors 1 

\-,sesslng skills/potential 1 

' sing resources effectively 1 

'instructing measurement tools 1 

nstllllng confidence 1 

. '«:< ivity (or awareness) 42 

Values of self and others 1 

Capabilities and limitations 3 

Needs of self and others 2 

Interactions of self and others J 

Worth in disciplines/methods 1 

.Vnlity In goal setting 1 

Degress of freedom to deviate 1 

Existing value systems 1 

Limitations of analyses/data 3 

Responses of target audiences b 

Sources of error 2 

Individual differences 1 

Recognition of data needs 3 

Response sets of target audiences 1 

Nature/scope of output 3 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




s 


Structure of Enabler 


Freq . 


IJV 


Primary Category of Enabler 




3 4 


Willingness to accept guidance 


1 


15 


Need to communicate effectively 


1 


37 


Willingness to delegate 


1 


39 


Limits of "one-shot" efforts 


1 


AO 


Contractual/stated obligations 


2 


4 7 


Willingness to work as needed 


2 


32 


Rccru 1 t ing/Pl ac ing Staff 


14 


1 


knowl edge 


51 


01 


Standard school subjects 


1 


03 


Subjects related to KDD&E 


2 


04 


Technlcal/professlot.' 1 topics 


3 


05 


Project focus topics, external 


2 


Of) 


Project variables: external 


10 


07 


Project operation: general 


1 


08 


Project operation: specific 


9 


09 


Scheduling and organizing 


4 


10 


Staff status/responslbll it ies 


3 


12 


Resources: personnel 


15 


16 


Guidelines for reporting 


1 


2 


Skill (or ability to perform) 


14 


02 


Facilitating people interactions 


1 


05 


Programing project events 


1 


08 


Analytical reading/study 


1 


14 


Writing 


1 


15 


Presenting orally 


1 


17 


Interpreting language 


1 


18 


Finding fits integrating 


1 


20 


Exercising Judgment 


1 


23 


Persuading/ Just if y ing 


6 


3 


Sensitivity (or awareness) 


21 


01 


Values of self and others 


2 


02 


Capabilities and limitations 


6 


03 


Needs of self and others 


L 


07 


Context of objectives 


1 
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Coding Sets 


Ouru.jt ci>d Work Requirement Categories 




DE 


t Category 




S 


of Enabler 


Freq . 


t)V 


Pci: Koiy Category of Enabler 






J C > U»ethod 


i 


i C 


Pxistirg value systems 


i 


i. t 


J'erser.rl Ity cf ethers 


i 




PotcVtiai ccniilict of interest 


i 




Tr* J I vidual differences 


i 




iccOfci.jLicn ; f data needs 


i 




.'v'c ittJi.g error/adapting 


i 




Pssponse sets of target audiences 


4 




‘ ‘ ~ r ? ^/Analyzing Data 


4 




Knowledge 


21 




Subjects related to RE\D&E 


4 


\ • > 


Technical/professional topics 


4 


v' 


Project variables: external 


1 




Project operation: general 


3 


V. c 


Project operation: specific 


3 


V % 


Scheduling and organizing 


1 


-• r 


Technical terminology/ language 


2 




Use of equipment/systems 


2 




Char's of target audience 


1 




■ l! 1 (or ability tu perform) 


15 


Cl 


Teaching 


1 


. 


Programing project events 


1 


Ui 


Programming technical equip 


2 


08 


Analytical reading/study 


1 


* / 


Interpreting language 


2 


^ . 


Fcrsuading/just if y ing 


1 


' 


Explicating goals/procedures 


1 


c 


Using equipment/systems 


3 


, f 


Getting others to perform 


1 


. 0 


Adapting to situation/demands 


1 


38 


Using resources effectively 


1 



Sensitivity (or awareness) 



6 



Interactions of self and others 



1 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




S 


Structure of Enabler 


Freq . 


UV 


Primary Category of Enabler 




19 


Supportiveness required 


i 


21 


Limitations of analyses/data 


i 


24 


Sources of error 


i 


36 


Feeling of personal security 


i 


38 


Emerging directions 


i 
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Illustrations of the Standards Associated with 
Primary Categories of Outputs 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Structure of Standards 


Freq . 


QR 


Primary Category of Standard 




01 


Reports/ Co ntrac ts 


26 


0006 


Terminal Report 


2 


1 


Output Standards 


11 


01 


Completeness of content 


1 


12 


Goal attainment 


1 


1A 


Acceptance by sponsor 


1 


15 


Compliance w/sponsor guideline 


1 


16 


Compares favorably 


3 


18 


Satisfactory appearance 


1 


19 


Logical criteria 


2 


2A 


Terminology appropriate 


1 


2 


Process Standards 


1 


20 


Performance respected 


1 


0007 


Proposal 


3 


1 


Output Standards 


5 


07 


Personal satisfaction/feeling 


1 


08 


Agreement /concurrence w/others 


1 


09 


Lack of errors/discrepancies 


1 


14 


Acceptance by sponsor 


1 


17 


Internally consistent 


1 


0158 


Evaluation Report 


5 


1 


Output Standards 


2A 


01 


Completeness of content 


1 


02 


Quantity of outputs/data 


1 


0A 


Communication and clarity 


1 


05 


Utility or value 


2 


08 


Agreement /concurrence w/others 


2 
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07 



j 



1 



( ( Vi*T ^ 1 Ti|P(| '/ 



I , ■ 



)u t p u t _ and Work Re q u i r ement Categories 
r Categoiy 

0jtp ,f t Category 
Structure of Standards 
Lt±HlU. Lacegory of Standard 



Freq , 



Lack of errors/discrepancies 2 

Appp p; lat* design/content 3 

Goal attainment 4 

by others (in proj ) 4 

Satisfactory appearance l 

Sources of variance controlled 2 

Meets design expectations 1 

Process Standards io 

Personnel are satisfied 1 

Deadlines are met 3 

Performance respected l 

Staff reflect trust l 

Impact of effort favorable 2 

Outputs distributed/requested 1 

Evidence of pre-planning 1 

, v i f ications/Procedures 21 

vipt ion/ Specs for Output 3 

Output Standards 8 

Completeness of content 1 

Communication and clarity 1 

Personal satisfaction/feeling 1 

Agreement/concurrence w/others 1 

Goal attainment l 

Acceptance by others (in proj) 1 

Internally consistent 2 

Data Analysis Procedures 3 

Output Standards 3 

Utility or value l 

Appropriate design/content 1 

Acceptance by others (in proj) l 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Structure of Standards 


Freq . 


QR 


Primary Category of Standard 




2 


Process Standards 


1 


14 


No felt deficiencies 


1 


0222 


Data Collection Procedures 


2 


1 


Output Standards 


8 


02 


Quantity of outputs/data 


1 


03 


Quantity of effort expended 


1 


04 


Communication and clarity 


1 


07 


Personal sat is fact ion /feeling 


1 


12 


Goal attainment 


1 


13 


Acceptance by others (in proj) 


1 


22 


Functions as planned 


2 


2 


Process Standards 


2 


24 


Costs acceptable for benefits 


1 


34 


Impact of effort favorable 


1 


27 


Data (including printouts) 


13 


0199 


Item Analysis Data 


2 


1 


Output Standards 


20 


01 


Completeness of content 


1 


04 


Communication and clarity 


2 


05 


Utility or value 


5 


06 


Acceptance by users 


1 


07 


Personal sat is fact ion /feeling 


2 


08 


Agreement /concurrence w/others 


1 


09 


Lack of errors/discrepancies 


1 


12 


Goal attainment 


1 


13 


Acceptance by others (in proj) 


3 


18 


Satisfactory appearance 


1 


19 


Logical criteria 


1 


28 


Operable by others 


1 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




EG..,. 


Primary Output Category 




IN' « 


Structure of Standards 


Freq . 




Primary Category of Standard 





Process Standards 



3 



No felt deficiencies 



3 



'■H I iue Data 



1 
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Illustrations of the Tasks Associated with 
Primary Categories of Outputs 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FCHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




01 


Reports/Contracts 


26 


0006 


Terminal Report 


2 


21 


Procuring professional staff 


1 


01 


Analyze work for competencies required 


1 


04 


Producing the output 


3 


04 


Secure/ select required materials/ comps 


1 


05 


Produce output 


2 


05 


Collecting/processing data 


1 


09 


Interpret data 


1 


01 


Clarifying Problem Addressed 


1 


12 


Review of /Familiarization w/materials 


1 


06 


Assessing the output quality 


2 


15 


Proof read/edit written material 


2 


0007 


Proposal 


3 


01 


Clarifying Problem Addressed 


3 


02 


Specify context/scope/limits of effort 


1 


06 


Observe relevant/related field opns 


1 


12 


Review of /Familiarization w/materials 


1 


02 


Formulating objectives 


3 


02 


Conceptualize the product and context 


1 


07 


Confer with colleagues/others re objects 


2 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




03 


Designing the output 


3 


• 03 


Determine relation of variables/compnt s 


1 


07 


Conceptualize output application 


1 


20 


State overall design 


1 


21 


Procuring professional staff 


5 


01 


Analyze work for competencies required 


1 


03 


Formulate job descriptions/qualif /salary 


2 


13 


Determine manpower needs 


1 


14 


Determine tasks to procure staff 


1 


0158 


Evaluation Report 


5 


01 


Clarifying Problem Addressed 


5 


03 


Specify variables to be studied 


2 


07 


Identify characteristics tgt population 


1 


12 


Review of /Familiarization w/materials 


2 


02 


Formulating objectives 


4 


03 


Formulate specific questions/hypotheses 


2 


04 


Determine nature of msmt objectives 


1 


07 


Confer with colleagues/others re objects 


1 


03 


Designing the output 


10 


03 


Determine relation of variables/compnts 


1 


05 


Specify/ select grouping/ sampling 


1 


06 


Specify treatments/procedures/ strategies 


3 


08 


Translate objectives/data into meaning 


1 


09 


Visualize/ specify format /appearance 


1 


10 


Specify/identify character of materials 


1 


17 


Specify measures to be used 


1 


19 


Specify kind of data analysis to be used 


1 


04 


Producing the output 


20 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




01 


Construct/assemble measurement tools 


7 


02 


Conetruct/aseemble components of output 


7 


05 


Produce output 


4 


13 


Construct/employ production tools/aids 


1 


18 


Provide materials to producers 


1 


05 


Collect ing/processing data 


10 


02 


Administer/ Implement data instruments 


1 


03 


Record/score/classify/code raw data 


2 


07 


Compile computational data 


1 


09 


Interpret data 


4 


11 


Try out output/component for operability 


1 


19 


Specify/identify data identity codes 


1 


06 


Assessing the output quality 


17 


03 


Determine achievement of objectives 


1 


10 


Confirm operability/validity of tools 


4 


11 


Judge effectiveness of data gathering 


1 


14 


Note/make advisable modif icat ion(s) 


6 


15 


Proofread/edit written material 


5 



07 


Diffusing the output 


2 


17 


Distribute dutput/component 


2 


21 


Procuring professional staff 


1 


10 


Select /Hire /Appoint personnel 


1 


22 


Effecting accountability 


17 


04 


Establish/Maintain time lines 


4 


10 


Report status as required/appropriate 


3 


12 


Explicate procedures/ strategies /duties 


1 


14 


Issue production orders 


2 


21 


Develop record keeping system 


1 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




22 


Est imate /Provide material /personnel 


i 


27 


Monitor /Determine cost-effectiveness 


i 


28 


Specify /Establish/ Adopt policies 


2 


30 


Obtain staff input regarding time/funds 


1 


31 


Participate in production efforts as req 


1 


23 


Procuring systems/services 


1 


05 


Determine tasks/services to be performed 


1 


24 


Effecting quality control 


10 


09 


Initiate/Direct quality control action 


1 


10 


Monitor progress/status of work 


2 


11 


Proof /critique products/performances 


3 


23 


Confirm accuracy of data inputs/outputs 


4 


25 


Maintaining job satisfaction 


2 


01 


Specify/explain agency/project position 


1 


03 


Recognize merit 


1 


26 


Facilitating growth of staff 


2 


06 


Specify ownership /authorship policies 


1 


07 


Encourage publication of important work 


1 


29 


Facilitating relationships 


16 


04 


Interact w/setting to elicit cooperation 


5 


14 


Interact with settingto collaborate 


1 


15 


Interact with setting to enhance 


8 


16 


Make materials/experiences available 


1 


17 


Provide foreign language translations 


1 


31 


Diffusing information within project 


4 


09 


Keep involved personnel informed generally 


1 


12 


Refer problem to appropriate specialists 


2 


15 


Advise technically within expertise 


1 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




32 


Diffusing information beyond project 


3 


04 


Report project at agency/other meetings 


2 


11 


Approve articles/papers for publication 


1 


04 


Work Specif icat ions /Procedures 


21 


0095 


Description/Specs for Output 


3 


01 


Clarifying Problem Addressed 


8 


02 


Specify context /scope/limits of effort 


1 


08 


Determine problems to be addressed 


1 


12 


Review of /Familiarization w/materials 


2 


15 


Operationally define terms /variables 


4 


02 


Formulating objectives 


1 


06 


Create objectives 


1 


03 


Designing the output 


6 


06 


Specify treatments /procedures /strategies 


2 


08 


Translate objectives/data into meaning 


1 


10 


Specify/identify character of materials 


2 


20 


State overall design 


1 


04 


Producing the output 


3 


02 


Construct/assemble components of output 


1 


05 


Produce output 


1 


14 


Adapt materials for local /specif ic use 


1 


06 


Assessing the output quality 


4 


14 


Note/make advisable modification (s) 


1 


15 


Proofread/edit written material 


3 


22 


Effecting accountability 


2 


02 


Delegate respon sibil it ies/ author it y 


1 


04 


Establish/Maintain time lines 


1 
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Coding Sets 


Output and Work Requirement Categories 





DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 





ERLC 



24 



10 
] 8 



0220 



01 



01 



03 



24 



04 



05 

08 



05 



05 

06 

07 

08 
09 

11 



06 



01 

08 



22 



02 



24 



Effecting quality control 2 

Monitor progress/status of work 1 

Monitor staff awareness of responsibility 1 

Data Analysis Procedures 3 

Clarifying Problem Addressed 2 

Review Problem-relevant literature/info 2 

Designing the output 1 

Visualize/Specify effect of output 1 

Producing the output 4 

Produce output 1 

Select/produce data processing/handling 3 

Collecting/processing data 17 

Translate data into processing form 2 

Perform analyses as specified/indicated 8 

Compile computational data 2 

Prepare supporting documentation 2 

Interpret data 1 

Try out output/component for operability 2 

Assessing the output quality 3 

Check output against specifications 1 

Determine inadequate output components 2 

Effecting accountability 1 

Delegate responsibilities/authority 1 

Effecting quality control 1 

Monitor personnel performance 
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Coding Sets 


Output and Work Requirement Categories 




0E 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 




26 


Facilitating growth of staf. 


3 


01 


Provide for guidance within project 


3 


29 


Facilitating relat ion ships 


1 


14 


Interact with setting to collaborate 


1 


31 


Diffusing information within project 


2 


12 


Refer problem to appropriate specialists 


2 


0222 


Data Collection Procedures 


2 


03 


Designing the output 


3 


05 


Specify/select grouping/ sampling 


1 


06 


Specify treatments /procedures/ strategies 


1 


23 


Consider/design alternative approaches 


1 


04 


Producing the output 


3 


13 


Construct /employ production tools/aids 


3 


05 


Collecting/processing data 


18 


01 


Conduct /operate program to be measured 


2 


02 


Administer/ Implement data instruments 


5 


03 


Record /score /classify/ code raw data 


3 


04 


Transfer raw data into storage form 


1 


05 


Translate data into processing form 


2 


06 


Perform analyses as specified/indicated 


3 


08 


Prepare supporting documentation 


1 


11 


Try out output/coraponent for operability 


1 


06 


Assessing the output quality 


4 


11 


Judge effectiveness of data gathering 


1 


14 


Note/make advisable modif ication (s ) 


2 


16 


Determine adeq of implementation process 


1 
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Coding Sets 


Output and Work Requirement Categories ~ ~ 


~ _ ■ - 


he 

FGHI 


Cluster Output Category 

Primary Output Category 




NO 

QR 


TaBk Cluster Category 

Primary Category of Task 


Freq . 


22 


Effecting accountability 


1 


02 


Delegate responsibilit ies/aut hori t y 


1 


23 


Procuring systems/services 


I 


07 


Arrange visitation/schedule as needed 


1 


24 


Effecting quality control 


2 


17 


Direct /request critiques from others 


2 


29 


Facilitating relationships 


9 


04 


Interact w/setting to elicit cooperation 


4 


06 


Explain/Prepare ratnle for proced to site 


1 


15 


Interact w/setting to enhance 


3 


16 


Make materials/experiences available 


1 


31 


Diffusing information within project 


1 


09 


Keep involved personnel informed generally 




27 


Data (including printouts) 


13 


0199 


Item Analysis Data 


2 


02 


Formulating objectives 


1 


02 


Conceptualize the produce and context 


1 


03 


Designing the output 


1 


23 


Consider/design alternative approaches 


1 


04 


Producing the output 


5 


02 


Construct/asserable components of output 


4 


05 


Produce output 


1 


05 


Collecting/processing data 


8 


03 


Record/score/classify/code raw data 

4£9 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output "Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


OK 


Primary Category of Task 




06 


Perform analyses as specified/indicated 


i 


09 


Interpret data 


L 


11 


Try out output/component for operability 


1 


21 


Procuring professional staff 


6 


11 


Provide training as appropriate 


6 


24 


Effecting quality control 


3 


06 


Monitor adherence to output specs 


1 


07 


Monitor personnel performance 


1 


17 


Direct/ request critiques from others 


1 


29 


Facilitating relationships 


1 


14 


Interact with setting to collaborate 


1 


30 


Effecting information flow patterns 


1 


03 


Insure that key decision points get info 


1 


31 


Diffusing information within project 


1 


09 


Keep involved personnel informed generally 


1 


33 


Effecting decision mechanisms 


1 


03 


Make decisions 


1 


0235 


Baseline Data 


1 


01 


Clarifying Problem Addressed 


2 


03 


Specify variables to be studied 


1 


12 


Review of /Familiarization w/materials 


1 


04 


Producing the output 


1 


13 


Construct /employ production tools/aids 


1 


05 


Collecting/processing data 


3 


02 


Administer/ Implement data instruments 


3 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




NO 


Task Cluster Category 


Freq . 


QR 


Primary Category of Task 





06 Assessing the output quality 3 



08 

7.3 

09 

29 

. 14 



Determine inadequate output components 3 

Procuring systeius/services 1 

Procure required tools/equipment 1 

Facilitating relation ships 2 

Interact with setting to collaborate 2 
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Illustrations of the Knowledges, Skills, and Sensitivities 
Associated with Primary Categories of Outputs 



Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




s 


Structure of Enabler 


Freq. 


uv 


Primary Category of Enabler 




01 


Reports /Contracts 


26 


0006 


Terminal Report 


2 


1 


Knowledge 


8 


02 


Subjects learned in courses 


1 


03 


Subjects related to RDD&E 


4 


07 


Project operation: general 


1 


12 


Resources: personnel 


1 


16 


Guidelines for reporting 


1 


2 


Skill (or ability to perform) 


2 


10 


Analytical data handling 


1 


14 


Writing 


1 


3 


Sensitivity (or awareness) 


1 


18 


Potential conflict of interest 


1 


0007 


Proposal 


3 


1 


Knowledge 


8 


05 


Project focus topics, external 


1 


06 


Project variables: external 


4 


08 


Project operation: specific 


1 


09 


Scheduling and organizing 


1 


11 


Fiscal matters 


1 


2 


Skill (or ability to perform) 


6 


14 


Writing 


1 


18 


Finding fits integrating 


2 


19 


Planning /conceptualizing 


1 


22 


Estimating expenses/resources 


1 


24 


Explicating goals/procedures 


1 



3 Sensitivity (or awareness) 
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, ' ■ 


Coding Sets 


Output and Work Requirement Categories 




DE 


Clustei Output Category 




FGHI 


Primary Output Category 




s 


Structure of Enabler 


Freq . 


UV 


Primary Category of Enabler 




02 


Capabilities and limitations 


1 


Oj 


Needs of self and others 


2 


04 


Interactions of self and others 


1 


07 


Context of objectives 


1 


no 


Awareness of alternatives 


1 


10 


Awareness of structure 


2 


0158 


Evaluation Report 


5 


1 


Knowledge 


33 


02 


Subjects learned in courses 


5 


03 


Subjects related to RDD&E 


12 


04 


Technical /professional topics 


1 


05 


Project focus topics, external 


2 


06 


Project variables: external 


6 


07 


Project operation: general 


1 


08 


Project operation: specific 


1 


12 


Resources: personnel 


1 


21 


Management techniques 


1 


22 


Use of equipment /systems 


1 


23 


Char’s of target audience 


1 


30 


Errors in strategies/ judgment 


1 


2 


Skill (or ability to perform) 


27 


01 


Teaching 


1 


02 


Facilitating people interactions 


4 


05 


Programming project events 


1 


09 


Analytical problem solving 


2 


10 


Analytical data handling 


3 


11 


Disciplining self 


1 


14 


Writing 


2 


17 


Interpreting language 


1 


18 


Finding fits intergrating 


1 


19 


Planning/conceptualizing 


2 


20 


Exercising judgment 


1 


24 


Explicating goals/procedures 


1 


25 


Applying measurement tools 


2 


29 


Getting others to perform 


1 
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Coding Sets 


Output and Work Requirement Categories 




DE 

FGHI 


Cluster Output Category 

Primary Output Category 




S 

uv 


Structure of Enabler 

Primary Category of Enabler 


Freq . 


30 


Adapting to situation/deraands 


i 


34 


Coordinating activities 


i 


35 


Communicating clearly 


i 


45 


Assessing personal performance 


i 


3 


Sensitivity (or awareness) 


24 


02 


Capabilities and limitations 


1 . 


03 


Needs of self and others 


2 


13 


Language barriers 


1 


17 


Personality of others 


1 


19 


Supportiveness required 


1 


20 


Unstated obligations 


1 


21 


Limitations of analyses /data 


1 


22 


Responses of target audiences 


1 


25 


Individual differences 


1 


30 


Response sets of target audiences 


3 


33 


Need for excellence in work 


2 


36 


Feeling of pe "onal security 


1 


38 


Limits of "one-shot" efforts 


1 


40 


Contractual/stated obligations 


1 


42 


Enthusiasm 


1 


44 


Interested in activities 


1 


55 


Respect for /thrust in others 


4 


04 


Work Specifications/Procedures 


21 


0095 


Description/Specs for Output 


3 


1 


Knowledge 


11 


02 


Subjects learned in courses 


1 


04 


Technical /professional topics 


7 


06 


Project variables: external 


1 


07 


Project operation: general 


1 


08 


Project operation: specific 


1 


2 


Skill (or ability to perform) 


7 


08 


Analytical reading/study 


1 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FGHI 


Primary Output Category 




s 


Structure of Enabler 


Freq . 


uv 


Primary Category of Enabler 




09 


Analytical problem solving 


1 


IS 


Finding fits integrating 


2 


20 


Exercising judgment 


1 


24 


Explicating goals/procedures 


1 


35 


Communicating clearly 


1 


3 


Sensitivity (or awareness) 


3 


01 


Values of self and others 


1 


10 


Awareness of structure 


1 


14 


Reality in goal setting 


1 


0220 


Data Analysis Procedures 


3 


1 


Knowledge 


17 


01 


Standard school subjects 


1 


02 


Subjects learned in courses 


2 


03 


Subjects related to RDD&E 


5 


04 


Technical/professional topics 


5 


08 


Project operation: specific 


1 


22 


Management techniques 


3 


2 


Skill (or ability to perform) 


6 


01 


Teaching 


2 


10 


Analytical data handling 


2 


11 


Disciplining self 


1 


19 


Planning/conceptualiz ing 


1 


3 


Sensitivity (or awareness) 


1 


21 


Limitations of analyses/data 


1 


0222 


Data Collections Procedures 


2 


1 


Knowledge 


5 


03 


Subjects related to RDD&E 


2 


06 


Project variables: external 


1 


21 


Manageraen t t echn ique s 


2 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




FOHI 


Primary Output Category 




S 


Structure of Enabler 


Freq . 


uv 


Primary Category of Enabler 




2 


Skill (or ability to perform) 


5 


02 


Facilitating people interactions 


3 


27 


Using equipment/systems 
Taking another’s perspective 


1 


31 


1 


3 


Sensitivity (or awareness) 


9 


02 


Capabilities and limitations 


1 


16 


Existing value systems 


1 


22 


Responses of target audiences 


3 


30 


Response sets of target audiences 


1 


31 


Nature/scope of output 


1 


35 


Need to communicate effectively 


1 


44 


Interested in activities 


1 


27 


Data (including printouts) 


13 


0199 


Item Analysis Data 


2 


1 


Knowledge 


14 


02 


Subjects learned in courses 


1 


03 


Subjects related to RDD&E 


5 


06 


Project variables: external 


3 


07 


Project operation: general 


1 


08 


Project operation: specific 


1 


22 


Use of equipment/systems 


1 


24 


Process implementation (project) 


2 


2 


Skill (or ability to perform) 


11 


01 


Teaching 


1 


05 


Programming project events 


1 


10 


Analytical data handling 


1 


14 


Writing 


2 


19 


Planning/conceptualizing 


2 


20 


Exercising judgment 


2 


26 


Locating/maintaining information 


1 


38 


Using resources effectively 
<- 411 
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Coding Sets 


Output and Work Requirement Categories 




DE 


Cluster Output Category 




E I 


Primary Output Category 




S 


Structure of Enabler 


Freq . 


uv 


Primary Category of Enabler 




3 


Sensitivity (or awareness) 


9 


04 


Interactions of self and others 


1 


06 


Worth in disciplines/methods 


1 


21 


Limitations of analyses/data 


4 


22 


Responses of target audiences 


2 


31 


Nature/scope of output 


1 


0235 


Baseline Data 


1 


1 


Knowledge 


6 


03 


Subjects related to RDD&E 


1 


05 


Project focus topics, external 


3 


06 


Project variables: external 


2 


3 


Sensitivity (or awareness) 


1 


26 


Recognition of data needs 


1 
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Personnel Characteristics and Activities that 
Differentiate RDD&E Job Roles 

Factor 1: Staff Worker (low-level data analysis and instrument development) 

Characteristics of Personnel with High Positive/Negative Factor Loadings 



Factor 






Highest 


Relationship 


Project 


Proj ec 


Load ing 


Person 


Sex 


Degree 


to Project 3 


Job Role* 3 


Focus 


.77 


20-02 


M 


Bach 


Reg. Staff 


Member, prof 


Res 


.71 


11-07 


F 


Bach 


Reg. Staff 


Member, prof 


Devel 


.68 


07-07 


M 


Bach 


Reg. Staff 


Member, prof 


Devel 


.68 


10-02 


F 


Bach 


Reg. Staff 


Member, tech 


Res 


.64 


11-06 


F 


Bach 


Reg. Staff 


Member, prof 


Devel 


. 63 


21-02 


F 


Mstr 


Reg. Staff 


Supervisor 


Dif f us 


.62 


12-04 


F 


Mstr 


Reg. Staff 


Member, prof 


Eval 


.58 


18-08 


r M 


Dr 


Reg. Staff 


Member, prof 


Eval 


.57 


11-08 


F 


Bach 


Reg. Staff 


Member, prof 


Devel 


.51 


02-07 


M 


Mstr 


Reg. Staff 


Member, prof 


Devel 


.40 


11-05 


F 


Bach 


Reg. Staff 


Member, prof 


Devel 


.38 


18-05 


F 


Bach 


Mgr /Sped 


Member, prof 


Eval 


.37 


14-08 


(background data 


not available) 




Res 


.36 


07-08 


F 


Bach 


Reg. Staff 


Member, prof 


Devel 


.32 


11-03 


M 


Dr 


Reg. Staff 


Manager 


Devel 


.30 


09-01 


M 


Dr 


Reg. Staff 


Prin Inves 


Res 


-.30 


13-06 


M 


Dr 


Adv/Consult 


Adv/Consult 


Devel 


-.31 


15-01 


M 


Dr 


Reg. Staff 


Prin Inves 


Devel 


-.33 


18-11 


M 


Mstr 


Reg. Staff 


Manager 


Eval 


-.34 


21-07 


F 


Bach 


Reg. Staff 


Member, prof 


Dif f us 


-.36 


21-01 


M 


Dr 


Mgr/Specl 


Prin Inves 


Dif f us 


-.37 


13-05 


M 


Mstr 


Coop Agency 




Devel 


-.39 


21-03 


M 


Mstr 


Mgr/Specl 


Manager 


Dif f us 


-.39 


02-05 


M 


Dr 


Reg. Staff 


Manager 


Devel 


-.40 


18-03 


M 


Mstr 


User Agency 


Manager 


Eval 


-.42 


08-10 


M 


Bach 


Reg. Staff 


Manager 


Dif f us 


-.46 


14-02 


M 


Bach 


Reg. Staff 


Manager 


Res 


-.46 


21-05 


F 


Bach 


On-call Staff 


Member, prof 


Dif f us 


-.50 


18-10 


M 


Mstr 


Reg. Staff 


Manager 


Eval 


-.54 


10-01 


M 


Dr 


Reg. Staff 


Prin Inves 


Res 


-.61 


19-01 


M 


Mstr 


Reg. Staff 


Prin Inves 


Dif f us 


-.61 


13-01 


M 


Mstr 


Reg. Staff 


Proj Dir 


Devel 


-.64 


17-02 


M 


Mstr 


Mgr/Specl 


Manager 


Dif fus 


-.67 


21-08 


M 


Dr 


Mgr/Specl 


Proj Dir 


Dif f us 


-.68 


17-05 


M 


Mstr 


Reg. Staff 


Proj Dir 


Dif fus 


-.80 


18-06 


M 


Mstr 


Reg. Staff 


Manager 


Eval 


-.83 


13-04 


M 


Mstr 


Reg. Staff 


Prin Inves 


Devel 


a See item 


46 on Form 


03 for 


full description of each < 


category . 





b See coding set X for full description of each category. 
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Activity Jtema DifferenLi *. lug Personnel 





Wit i 


t High Positive/Negative Factor Loadings 


Sc a J r 


Form 04 




Difference 


No . 


Label for Activity Statement 


4.7 


E. 2 


Computing or using measures of central tendency (i.e., 
means, medians, modes, arithmetic average). 


4.1 


E.l 


Preparing or using frequency tallies and/or marginal 
distributions (as in Chi-Square tests). 


3.9 


E. 3 


Computing or using correlation coefficients, including 






simple correlational analyses. 


3.4 


E. 8 


Computing item analyses of test items. 


3.3 


E. 5 


Computing and interpreting data from analysis of 
variance designs. 


3.2 


E.4 


Computing and interpreting simple tests of significance 
of differences in observed data (such as t-tests). 


3.0 


E. 7 


Examining and interpreting non-quantif ied information 
(such as verbal responses, observed activities, etc.). 


2.9 


C . 2 


Developing test items. 


2.6 


E. 


Analyzing data 


2.4 


E. 6 


Computing and interpreting regression analyses. 


2.2 


C. 


Developing research tools or other info, .nation-gathering 
instruments . 


2.2 


e.l 


Constructing questionnaires. 


-2.1 


H.4 


Designing appropriate learning situations. 


-2.1 


F.4 


Writing interim, status, or periodic reports. 


-2.2 


B.l 


Identifying relevant variables for consideration. 


-2.3 


F. 3 


Writing major project reports. 


-2.5 


F. 6 


Writing administrative reports. 


-2.8 


G.4 


Communicating personnel evaluations to individuals. 


-2.8 


G. 2 


Establishing contact with and participation by other 
personnel or agencies. 


-2.8 


G. 5 


Scheduling project activities. 


-3.0 


I.] 


Contacts with funding sponsor or monitor of project. 


-3.2 


G. 


Supervising and coordinating actions of others, and/or 
of material resources. 


-3.2 


B. 2 


Developing conceptual "rameworks or general patterns 
of project design. 


-3.5 


1.2 


Contacts with higher agency management for review of 
proj ec t . 


-3.6 


B. 4 


Organizing a coherent program of activities. 


-3.6 


G.l 


Procurement of project staff. 


-4.0 


G . 3 


Reviewing performance of project personnel. 


-4 . 6 


G. 6 


Allocating responsibilities to project personnel. 



Factor 2: Staff Worker (high-level statistical design, and instrument design) 

Characteristics of Personnel with High Positive/Negative Factor Loadings 



Fac tor 






Highest 


Relationship 


Project 


Proj ect 


Loading 


Person 


Sex 


Degree 


to Project 


Job Role 


Focus 


.63 


18-05 


F 


Bach 


Mgr/Specl 


Member, prof 


Eval 


.43 


16-02 


F 


Dr 


Mgr/Specl 


Member, prof 


Eval 


.37 


11-03 




J- " r 4 $ 


Reg. Staff 


Manager 


Devel 



APPENDIX 19 
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.34 


07-03 


M 


Dr 


Reg . 


Staff 


Manager 


Devel 


. 32 


11-08 


F 


Bach 


Reg. 


Staff 


Member, prof 


Devel 


-.42 


10-01 


>1 


Dr 


Reg.. 


Staff 


Prin Inves 


Res 


30 


21-07 


K 


Bach 


Reg. 


Staff 


Member, prof 


Dif f us 





Activity Items Differentiating Personnel 




With 


High Positive/Negative Factor Loadings 


Seal e 


Form 04 




Difference 


No . 


Label for Activity Statement 


5 . 6 


B. 5 


Designating sampling procedures. 


5.6 


E. 5 


Computing and interpreting data from analysis of 
variance designs. 


5.2 


E.4 


Computing and interpreting simple tests of significance 
of differences in observed data (such as t-tests). 


5.1 


C . 4 


Developing observational techniques. 


5.0 


E. 3 


Computing or using correlation coefficients, including 
simple correlational analyses. 


4.6 


E. 2 


Computing or using measures of central tendency (i,e., 
means, medians, modes, arithmetic average). 


4.4 


E.l 


Preparing or using frequency tallies and/or marginal 
distributions (as in Chi-Square tests). 


4.3 


E. 7 


Examining and interpreting non-quan tif ied information 
(such as verbal responses, observed activities, etc.). 


4.0 


E. 6 


Computing and interpreting regression analyses. 


3.9 


B. 6 


Designating general statistical treatment to be used. 


3.7 


B. 7 


Designing system models for computer application 
to data. 


3.5 


C. 


Developing research tools or other information-gathering 
instruments . 


3.0 


E . 8 


Computing item analyses of test items. 


2.9 


B. 3 


Developing methodologies to be used in the project. 


2.6 


B. 


Designing or planning procedural activities for the 
project . 


2.5 


C.l 


Cons true ting questionnaires . 


2.5 


C . 2 


Developing test items. 


2.4 


C. 3 


Developing interview outlines and schedules. 


2.0 


F. 


Writing . 


-2.2 


F.4 


Writing interim, status, or periodic reports. 


-2.3 


F.l 


Writing correspondence. 


-2.4 


A. 4 


Reading "in-house" materials and correspondence. 


-2.7 


H. 3 


Providing on-the-job training to individuals. 


-3.0 


F . 3 


Writing major project reports. 


-3.2 


F. 7 


Writing literature surveys. 


-3.4 


A. 1 


Reading recent project-related research. 


-3.4 


1.2 


Contacts with higher agency management for review 
of project. 


-3.5 


G. 5 


Scheduling project activities. 


-3.6 


F. 6 


Writing administrative reports. 


-3.6 


C.l 


Procurement of project staff. 


-3.7 


C . 4 


Communicating personnel evaluations to individuals. 


-3.8 


G.2 


Establishing contact with and participation by other 
personnel or agencies. 
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-h . '» 


<; . (i 


Allocating responsibilities to p 


reject persouue 


i . 


~4 . U 


A . 2 


Read Lng scholarly essays . 






-4 . / 


F. J 


Writing research proposals. 






-4 , ri 


v . :> 


Writing for 


professional publ icar ions . 




-5.4 


G . 3 


Reviewing performance of project 


personnel . 




Factor 3: 


Supervisor (high-level data analysis) 






Characteristics of 


Personnel with High Positivc/Negative Factor Loadings 


Factor 




highest 


Relationship 


Pro j ec t 


Proj ect 


loading 


i\ rson 


Sex Degree 


to Project 


Job Role 


Focus 


.79 


10-03 


M Dr 


Reg. Staff 


Proj Dir 


Res 


.45 


18-09 


M Dr 


Reg. Staff 


Supervisor 


Eval 


.44 


10-02 


F Bach 


Reg. Staff 


Tech Support 


Res 


.30 


16-01 


M Dr 


Mgr / Sped 


Proj Dir 


Eval 


.30 


21-02 


F Mstr 


Reg. Staff 


Supervisor 


Diffus 


-.32 


17-11 


F Mstr 


Reg. Staff 


Member, prof 


Diffus 


-.51 


17-12 


F Mstr 


Reg. Staff 


Member, prof 


Diffus 


-.54 


15-02 


M Ms t r 


Reg. Staff 


Manager 


Devel 


-.57 


17-10 


M Mstr 


Reg. Staff 


Member, prof 


Diffus 


-.71 


07-14 


M Mstr 


Particip Agency 


Manager 


Devel 




Activity Items 


Differentiating Personnel 






With High Positive/Negative Factor Loadings 




Scale 


Form 


04 








Dif ferenct 


NO . 


Label for 


Activity Statement 







4 . 0 


E. 3 


4.6 


E.4 


4.4 


E . 5 


4.0 


E. 2 


3.8 


E. 6 


3.0 


11. 7 


3.4 


B.6 


3.4 


E.l 


3.2 


E. 8 


2.8 


G.l 


2.6 


G.6 


2.4 


G . 4 


2.0 


G. 3 


-2.2 


1.2 


-2.4 


A. 1 


-2.6 


A. 



Computing or using correlation coefficients, including 
simple correlational analyses. 

Computing and interpreting simple tests of significance 
of differences in observed data (such as t-tests). 
Computing and interpreting data from analysis of 
variance designs. 

Computing or using measures of central tendency (i.e., 
means, medians, modes, arithmetic average). 

Computing and interpreting regression analyses. 

Designing system models for computer application to data. 
Designating general statistical treatment to be used. 
Preparing or using frequency tallies and/or marginal 
distributions (as in Chi-Square tests). 

Computing item analyses of test items. 

Procurement of project staff. 

Allocating responsibilities to project personnel. 
Communicating personnel evaluations to individuals. 
Reviewing performance of project personnel. 

Contacts with higher agency management for review 
of project. 

Reading recent project-related research. 

Reading . 
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-2.6 


B . 4 


Organizing a coherent program of activities. 


-2.6 


D . 3 


Conducting laboratory experiments. 


-2.6 


D . 4 


Administering questionnaires. 


-2.6 


F . 4 


Writing interim, status, or periodic reports. 


-2.6 


F . 6 


Writing administrative reports. 


-2.8 


0.2 


Surveying literature. 


-2.8 


0.3 


Administering tests . 


-2.8 


11.2 


Participating in conduct of seminars of workshops. 


-3.0 


C. 3 


Developing interview outlines and schedules. 


-3.0 


C . 4 


Developing observational techniques . 


-3.2 


D. 


Collecting project data. 


-3.2 


C . 2 


Developing test items. 


-3.2 


F . 3 


Writing major project reports. 


-3.4 


H. 


Teaching or training. 


-3.4 


C.l 


Constructing questionnaires 


-3.6 


D.l 


Interviewing. 


-3 . 6 


H. 1 


Participating in classroom instruction. 


-4.0 


C. 


Developing research tools or other inf ormat ion-gathering 






instruments . 


-5.2 


H. 4 


Designing appropriate learning situations. 


Factor 4: 


Manager 


(mid-management, with staff personnel contact) 



Characteristics of Personnel with High Positive/Negative Factor Loadings 



Factor 






Highest 


Relationship 


Proj ect 


Proj ee 


Loading 


Person 


Sex 


Degree 


to Proiect 


Job Role 


Focus 


.88 


07-05 


M 


Bach 


Reg. Staff 


Manager 


Devel 


.56 


02-05 


M 


Dr 


Reg. Staff 


Manager 


Devel 


.44 


07-13 


F 


Mstr 


Reg. Staff 


Member, prof 


Devel 


.35 


13-05 


M 


Mstr 


Partic Agency 


Adv/Consult 


Devel 


.32 


13-03 


M 


Mstr 


Par tic Agency 


Adv/Consult 


Devel 


.32 


13-01 


M 


Mstr 


Reg. Staff 


Proj Dir 


Devel 


.31 


16-04 


M 


Dr 


Reg. Staff 


Supervisor 


Eval 


-.34 


20-01 


M 


Dr 


Reg. Staff 


Prin Inves 


Res 


-.35 


12-02 


M 


Mstr 


Reg. Staff 


Manager 


F.vai 


-.51 


16-02 


F 


Dr 


Mgr/ Sped 


Member, prof 


Eval 


-.54 


14-01 


M 


Dr 


Reg. Staff 


Prin Inves 


Res 



Activity Items Differentiating Personnel 





With 


High Positive/Negative Factor Loadings 


Scale 


Form 04 




Difference 


No . 


Label for Activity Statement 


5.0 


F . 3 


Writing major project reports. 


4.0 


A. 4 


Reading M in-house H materials and correspondence. 


3 4 


G.4 


Communicating personnel evaluations to individuals. 


3.4 


1.1 


Contacts with funding sponsor or monitor of project. 


3.2 


G . 6 


Allocating responsibilities to project personnel. 


2.8 


G 


Supervising and coordinating actions of others, and/or 



of material resources. 

k ; r. 

•T - . • 



ERLC 
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. h 
i 
i 


l 

ii. 7 # 


Reviewing performance of project personnel. 
Procurement of project staff. 

Designing appropriate Learning situations. 


-2.1 


E. 2 


Co, .puling or using Measures of central tendency 
(i.c., means, medians, nodes, arithmetic average). 


-2. 3 


K. 7 


Examining and interpreting non-quant if ied information 
(such as verbal responses, observed activities, etc.). 


- 2 . ?) 


K. 


Writing 


-2.5 


L.3 


Computing or using correlation coefficients, including 
simple correlational analyses. 


-L\f> 




Developing methodologies to be used in the project. 


-2.6 


2.3 


Developing interview outlines and schedules. 


- 2.7 


A. 3 


Heading methodological documents presenting information 
regarding methods of inquiry and/or analysis. 


-2.7 


fc.4 


Computing and interpreting simple tests of significance 
of differences in observed data (such as t-tests). 


-2. / 


F. 5 


Writing for professional publications. 


-2.8 


13.2 


Developing conceptual frameworks or general patterns 
of project design. 


-2.9 


E.l 


Preparing or using frequency tallies and/or marginal 
distributions (as in Chi-Square tests). 


-3.1 


L. 


Analyzing data. 


-3.1 


C.l 


Constructing questionnaire. 


-3.3 


C. 


Developing research tools or other information- 
gathering instruments . 


-3. / 


B.l 


Identifying relevant variables for consideration. 


-3.8 


B. 7 


Designing system models for computer application 
to data. 


-3.8 


F.4 


Writing interim, status, or periodic reports. 


-A . 0 


F. 3 


Writing major project reports. 


-5.2 


B. 5 


Designating sampling procedures. 


-5.4 


B. 6 


Designating general statistical treatment to be used. 



Tnct^r 5: Management Representative (outside contacts) 

Characteristics of Personnel with High Positive/Negative Factor Loadings 



Fac tor 



Loading 


Person 


Sex 


.90 


13-02 


M 


.49 


19-02 


F 


.40 


08-12 


M 


.33 


12-01 


M 



-.32 


21-06 


M 


-.32 


09-03 


F 


-.34 


18-07 


F 


-.46 


20-01 


M 


-.50 


09-01 


M 


-.55 


09-04 


M 



Highest 

Degree 


Relationship 
to Project 


Mstr 

Mstr 

Dr 

Dr 


Mgr/ Sped 
Reg Subcontract 
Reg. Staff 
ReR. Staff 


Dr 


Adv/Ccnsult 


Mstr 


Reg. Staff 


Mstr 


Reg. Staff 


Dr 


Reg. Staff 


Dr 


Reg. Staff 


Mstr 


On-call Staff 



Proj ect 


Proj ec t 


Job Role 


Focus 


Adv/ Consult 


Devel 


Members, prof 


Dif f us 


Manager 


Dif f us 


Prin Inves 


Eval 


Manager 


Dif fus 


Member, prof 


Res 


Supervisor 


Eval 


Prin inves 


Res 


Prin Inves 


Res 


Tech Support 


Res 
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Scale 


Form i 


Difference 


No . 


5.7 


1.2 


5.2 


1.1 


4.9 


1.3 


3.8 


1.4 


2.2 


A. 


-2.0 


B. 2 


-2.0 


D . 4 


-2.1 


B. 3 


-2.1 


B. 6 


-2.2 


D. 2 


-2.2 


D . 5 


-2.6 


B. 5 


-2.6 


E. 8 


-2.8 


C.4 


-2.9 


B.l 


-2.9 


B. 4 


-3.0 


B. 


-3.0 


C.l 


-3.2 


C. 


-3.4 


H. 4 


-3.8 


C . 3 


-4.1 


D.l 


-5.0 


D. 


Factor 6: 


Staff 



Activity Items Differentiating Personnel 
With High Positive/Negative Factor Loadings 



Label for Activit y Staten *nt 

with higher agency management for review of 



Contacts 
project . 

Contacts with funding sponsor or monitor of project. 
Presentations made at professional meetings to 
communicate various aspects of project activities 
or results. 

Meeting with visiting personnel from other agencies. 

Reading . ^ 

Developing conceptual frameworks or general patterns 
of project design. 

Administering questionnaires. 

Developing methodologies to be used in the project. 
Designating general statistical treatment to be used. 
Surveying literature. 

Administering tests . 

Designating sampling procedures. 

Computing item analyses of test items. 

Developing observat ional techniques. 

Identifying relevant variables for consideration. 
Organizing a coherent program of activities. 

Designing or planning procedural activities for the projec 
Constructing questionnaires . 

Developing research tools or other information-gathering 
instruments . 

Designing appropriate learning situations. 

Developing interview outlines and schedules. 

Interviewing . 

Collecting project data. 



Characteristics of Personnel with High Positive/Negative Factor Loadings 



Factor 






Highest 


Relationship 


Proj ect 


Proj ect 


Loading 


Person 


Sex 


Degree 


to Proiect 


Job Role 


Focus 


.80 


19-04 


F 


Mstr 


Reg. Staff 


Member, prof 


Dif fus 


.78 


19-05 


F 


Bach 


Reg. Staff 


Member, prof 


Dif fus 


.68 


19-06 


F 


Mstr 


Reg. Staff 


Member, prof 


Dif fus 


.50 


15-C4 


M 


Dr 


Reg. Staff 


Mana 6 er 


Devel 


.42 


21-06 


M 


Dr 


Adv/Consult 


Manager 


Dif fus 


.33 


07-13 


F 


Mstr 


Reg. Staff 


Member, prof 


Devel 


.32 


15-02 


M 


Mstr 


Reg. Staff 


Proj Dir 


Devel 


.30 


21-04 


F 


Mstr 


R°.g. Staff 


Supervisor 


Dif fus 


-.30 


17-11 


F 


Mstr 


Reg. Staff 


Member, prof 


Dif fus 


-.34 


11-04 


M 


Bach 


Reg. Staff 


Member, prof 


Devel 


-.35 


18-10 


M 


Mstr 


Reg. Staff 


Manager 


Eval 


-..0 


07-03 


M 


Dr 


FtqrpiJfaEf 


Manager 


Devel 
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Activity Items Differentiating Personnel 
With High Positive/ Negative Factor Loadings 

Scale Form 04 

Difference No . Label for Activity Statement 

3.9 A. 2 Reading scholarly essays. 

3.6 A. Reading. 

2.5 A. 4 Reading "in-house" materials and correspondence. 

2.4 D.2 Surveying literature. 

2.4 G.l Procurement of project staff. 

2.2 A. 3 Reading methodological documents presenting information 

regarding^ methods of inquiry and/or analysis. 

E. 3 Computing or using correlation coefficients, including 

simple correlational analyses. 

B.6 Designating general statistical treatment to be used. 

B.5 Designating sampling procedures. 

B. 2 Developing conceptual frameworks or general patterns 

of project design. 

F. 4 Writing interim, status, or periodic reports. 

C. l Constructing questionnaires. 

G. 2 Establishing contact with and participation by other 

personnel or agencies. 

£. Designing or planning procedural activities for the proj 

A. l Reading recent proj ec t-related research. 

B. 3 Developing methodologies to be used in the project. 

C. Developing research tools or other information-gathering 
instruments . 

B.4 Organizing a coherent program of activities. 

E. Analyzing data. 

B.l Identifying relevant variables for consideration. 



- 2.0 

- 2.0 

- 2,1 

- 2.2 

- 2.2 

-2.5 

- 2.6 

-2.9 

-3.0 

-3.1 

-3.1 

-3.2 

-3.8 

-3.9 



Factor 7: Undefined (possibly "team leader") 

Characteristics of Personnel with High Positive/Negative Factor Loadings 



Factor 

Loading 


Person 


Sex 


Highest 

Degree 


Relationship 
to Project 


Proj ec t 
Job Role 


Project 

Focus 


.82 


19-03 


M 


Bach 


On-call Staff 


Staff, prof 


Dif fus 


.60 


19-02 


F 


Ms tr 


Reg Subcontractor 


Staff, prof 


Dif fus 


.51 


21-03 


M 


Ms tr 


Mgr/ Sped 


Manager 


Dif fus 


.38 


18-09 


M 


Dr 


Reg. Staff 


Supervision 


Eval 


-.36 


21-06 


M 


Dr 


Adv/Consult 


Manager 


Dif fus 



Activity Items Differentiating Personnel 
With High Positive/Negative Factor Loadings 



O 

ERIC 



Scale 

Difference 

3.8 



Form 04 
No. 

E. 



Label for Activity Statement 
Analyzing data. 
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3.5 


G . 5 




Scheduling proiect activities. 




(-2.2 to - 


3.5) 




17 activities unlisted here. 




-4.0 


13. 3 




Developing methodologies to be used in the 


pro j ec t . 


-4.0 


8.4 




Organizing a coherent program of activities 


. 


-4.0 


E. 4 




Computing and interpreting simple tests of: 


significance 








of differences in observed data (such as t- 


tests). 


-4.0 


E. 5 




Computing and interpreting data from analysis of 








variance designs. 




-4.0 


E. 7 




Examining and interpreting non-quant if ied information 








(such as verbal responses, observed activities, etc.) 


-4.2 


13.1 




Identifying relevant variables for consideration. 


-4.5 


B. 2 




Developing conceptual frameworks or general patterns 








of project design. 




-4.5 


E. 8 




Computing item analyses of test items. 




-4.5 


H.4 




Designing appropriate learning situations. 




-5.0 


A. 1 




Reading recent project-related research. 




-5.2 


C . 2 




Developing test items. 




-5.2 


C . 3 




Developing interview outlines and schedules. 


-5.2 


C . 4 




Developing observational techniques. 




-5.8 


A. 3 




Reading methodological documents presenting 


information 








regarding methods of inquiry and/or analysis. 


-6.0 


D. 2 




Surveying literature. 




Factor 8: 


Information 


i Disseminator 




Characteristics of 


Personnel with High Positive/Negative Factor Loadings 


Fac tor 






Highest Relationship Project 


Proj ec t 


Loading 


Person 


Sex 


Degree to Project Job Role 


Focus 


.36 


17-09 


M 


Mstr Reg. Staff Member, prof 


Dif fus 


.31 


15-04 


M 


Dr Reg. Staff Manager 


Devel 


-.37 


09-04 


M 


Mstr On-Call Staff Tec Support 


Res 


-.55 


18-12 


M 


Dr Mgr/Specl Member, prof 


Eval 


-.85 


18-02 


M 


Dr Reg. Staff Manager 


Eval 




Activity Items Differentiating Personnel 






With High Positive/Negative Factor Loadings 




Scale 


Form 04 






Difference No. 




Label for Activity Statement 




5.5 


H.l 




Participating in classroom instruction. 




5.2 


H. 2 




Participating in conduct of seminars or workshops. 


4.8 


D.l 




Interviewing. 




4.8 


F . 5 




Writing for professional publications. 




4.7 


1.3 




Presentations made at professional meetings to 








communicate various aspects of project activities 








or results. 




3.8 


A. 2 




Reading scholarly essays. 




3.8 


F.l 




Writing correspondence. 




3.7 


H.4 




Designing appropriate 1 earning situat Ions . 





42.6 

«• - 
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(Continued) 



5. i 


1 . 1 


Contacts with funding sponsor or monitor of project. 


i . ') 


1) . 2 


Surveying literature . 


i . 3 


\), i 


Conduc t ing laboratory ex per linen t s . 


3.2 


i). 


Collecting project data. 


3 .0 


1:; . / 


Examining and interpreting non-quant i f Led inlurmaiion 
(such as verbal responses, observed activities, etc.). 


3.0 


1.4 


Meeting with visiting personnel from other agencies. 


2.8 


E . 


Analyzing data. 


2.8 


D .4 


Administering questionnaires. 


2.5 


F . 7 


Writing literature surveys. 


2.5 


G.l 


Procurement of project staff. 


2.3 


G.4 


Communicating personnel evaluations to individuals. 


2.2 


G . 5 


Scheduling project activities. 


2.0 


A.l 


Reading recent project-related research. 


2.0 


B . 2 


Developing conceptual frameworks or general patterns 
of project design. 


-2.0 


B. 7 


Designing system models for computer application to data 


-2.2 


F . 6 


Writing administrative reports. 


-2.7 


C. 


Developing research tools or other information-gathering 
instruments . 


-2.7 


E.8 


Computing item analyses of test items. 


-3.0 


C . 2 


Developing test items. 


-5.3 


C.l 


Constructing questionnaires. 


Factor 9: 


Undefined 


(possibly "research aide") 



Characteristics of Personnel with High Positive/Negative Factor Loadings 



Factor 






Highest 


Relationship 


Proj ec t 


Projec 


Loading 


Person 


Sex 


Degree 


to Project 


Job Role 


Focus 


.50 


07-13 


F 


Mstr 


Reg. Staff 


Member, prof 


Devel 


.32 


02-07 


M 


Mstr 


Reg. Staff 


Member, prof 


Devel 


-.31 


21-08 


M 


Dr 


Mgr/Specl 


Proj Dir 


Dif fus 


-.41 


21-06 


M 


Dr 


Adv/Consult 


Manager 


Dif fus 


-.49 


21-03 


M 


Mscr 


Mgr/ Sped 


Manager 


Dif fus 


-.84 


02-02 


F 


Dr 


Reg. Staff 


Proj Dir 


Devel 



O 

ERIC 



Activity Items Differentiating Personnel 
With High Positive/ Negative Factor Loadings 



Scale Form 04 

Difference No . Label for Activity Statement 



4.0 

3.2 

2.2 

2.0 

2.0 



- 2.0 

- 2.0 



D. Collecting project data. 

E. Analyzing data. 

A, 4 Reading "in-house" materials and correspondence. 

E.2 Computing or using measures of central tendency 

(i.e., means, medians, modes, arithmetic average). 

G. 2 Establishing contact with and participation by other 

personnel or agencies. 

D.3 Conducting laboratory experiments. 

H. 4 ^Designing appropriate learning situations. 

427 
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\, 



-2.2 


A . 1 


Reading recent project-related research. 


-2.2 


C. 3 


Developing interview outlines and schedules. 


-2.2 


F. 7 


Writing literature surveys. 


-2.2 


i . 4 


Meeting with visiting personnel from other agencies. 


-2.5 


B . 3 


Developing methodologies to be used in the project. 


-2.5 


D.l 


Interviewing . 


-2.8 


B . 5 


Designating sampling procedures. 


-2.8 


G . 3 


Reviewing performance of project personnel. 


-3.0 


F . 6 


Writing administrative reports. 


-3.0 


1.1 


Contacts with funding sponsor or monitor of project. 


-3.0 


1 . 2 


Contacts with higher agency management for review of 
proj ect . 


-3.2 


I. 


Meeting, consulting, or advising. 


-3.5 


D. 2 


Surveying literature. 


-3.5 


F . 2 


Writing research proposals. 


-3.8 


F . 4 


Writing interim, status, or periodic reports. 


-4 . 2 


G. 


Supervising and coordinating actions of others, and/or 
of material resources. 


-4 . 2 


F. 3 


Writing major project reports. 


-4.5 


G.5 


Scheduling project activities. 


-4.5 


G. 6 


Allocating responsibilities to project personnel. 


-4.8 


H. 3 


Providing on-the-job training to individuals. 


-5.0 


B.l 


Identifying relevant variables for consideration. 


-5.0 


B. 2 


Developing conceptual frameworks or general patterns 
of project design. 


-5.3 


B. 


Designing or planning procedural activities for the project 


-7.0 


B.4 


Organizing a coherent program of activities. 



Factor 10: Undefined (possibly "new doctorate-level employee with 

statistical skills") 

Characteristics of Personnel with High Positive/Negative Factor Loadings 



Fac tor 
Load ing 

.33 



Person 



16.04 



Sex 



Highest 

Degree 

Dr 



Relationship 
to Projec t 

Reg. Staff 



Proj ect 
Job Role 

Supervisor 



Proj ect 
Focus 

Eval 



.34 10-01 M Dr 

.82 11-01 M Dr 



Reg. Staff 
Reg. Staff 



Prin Inves 
Prin Inves 



Res 

Devel 



O 

ERIC 



Scale 

Difference 

7.0 

7.0 

7.0 

7.0 

6 . 0 



Activity Items Differentiating Personnel 
With High Positive/Negative Factor Loadings 



Form 04 
No. 

D . 3 

E. 2 

E.4 

11.4 

l). 



Label for Activity Statement 

Conducting laboratory experiments. 

Computing or using measures of central tendency 
(i.e., means, medians, modes, arithmetic average). 
Computing and interpreting simple tests of significance 
of differences in observed data (such as t-tests). 
Designing appropriate learning situations. 

Collecting project data. 428 
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6.0 C.l 

6.0 C . 4 

6.0 E . 5 

5.0 C. 



.0 C. 2 

.0 E. 3 

5.0 E. 6 

4.0 B. 5 

4.0 D.l 

(3.5 to 2.0) 

-2.0 E. 7 



-2.5 

-2.5 

-3.0 

-3.5 

-3.5 

-5.0 

-5.5 

-5.5 

- 6.0 



B.4 
G . 6 
F . 5 
A. 2 
F . 2 

F. l 

G. l 
G . 3 
F. 3 



(Continued) 

Constructing questionnaires. 

Developing observational techniques. 

Computing and interpreting data from analysis of 
variance designs. 

Developing research tools or other information-gathering 
instruments . 

Developing test items. 

Computing or using correlation coefficients, including 
simple correlational analyses. 

Computing and interpreting regression analyses. 
Designating sampling procedures. 

Interviewing. 

12 activities unlisted here. 

Examining and interpreting non-quantif ied information 
(such as verbal responses, observed activities, etc.). 
Organizing a coherent program of activities. 

Allocating responsibilities to project personnel. 

Writing for professional publications. 

Reading scholarly essays. 

Writing research proposals. 

Writing correspondence. 

Procurement of project staff. 

Reviewing performance of project personnel 
Writing major project reports. 
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A GUIDE TO THE OREGON STUDIES IN EDUCATIONAL RDD&E 



Volume I 

SUMMARY REPORT 

An introduction to and overview of the Oregon Studies as a 
whole. The volume contains an outline of the history of the 
Studies, the rationale around which they were designed, the 
context within which ^they were carried out, and the 
procedures followed in^ their execution. It also contains a 
description of the projects selected for study, the rationale 
underlying their selection, the criteria and procedures used 
in their selection, and an overview of the data collected on 
each project. Finally, the volume contains an introduction 
to the “case profiles*' that house the data collected on each 
project, the results of all cross-project analyses, and the 
summary recommendations that have been made relative to 
training and the continued study of educational RDD&E 
activities. A brief description of the case study method- 
ology developed within the Studies, an overview of a 
process whereby investigators may query computer-stored 
data files and original interview statements to obtain 
information bearing upon specific questions relating to 
training, manpower, policy, and work performance, and 
supporting data accompany the volume. 



Volume II 

THE LITERATURE OF EDUCATIONAL RDD&E 

A compendium of existing literature that defines, describes, 
differentiates, or relates the activities labeled educational 
research, development, diffusion, evaluation, and various 
combinations thereof. The articles within the volume are 
introduced as a collection. Linking passages provide an 
interpretive context both for individual articles and for the 
sets into which they have been grouped. 



Volume III 

CONCEPTUAL FRAMEWORKS FOR VIEWING 
EDUCATIONAL RDD&E 

A collection of papers which provide the conceptual 
underpinnings to the Oregon Studies. It contains three 
papers commissioned by the Studies as a basis for con- 
ceptual development, and a paper by staff from Teaching 
Research that describes the conceptual frame that guided 
and grew with the empirical thrust of the Studies. Each of 



the papers is a major document which defines, differ- 
entiates, and relates one or more facets of educational 
RDD&E and provides a supporting rationale for the 
position adopted. Each paper is accompanied by a formal 
critique, and the set of papers is accompanied by an 
introductory and summary critique. 

Volume IV 

PROFILES OF EXEMPLARY PROJECTS 
IN EDUCATIONAL RDD&E 

A collection of twenty case profiles that form the data 
base in the Oregon Studies. Printed in three parts, the 
profiles describe five research projects, seven development 
projects, thrfee evaluation projects, and five diffusion 
projects. Each profile contains descriptions of the structure 
and function of the project being analyzed, the specific 
outputs expected to emerge from it, the operations 
required to produce each output, and the knowledges, 
skills, and sensitivities judged to be essential to the 
performance of those operations. In addition, each profile 
contains sections dealing with the “dynamics'* of project 
operations and implications that derive from the project for 
preservice staff training. The projects described range from 
small, two-man efforts within university settings to very 
large school district “projects" employing several dozen 
staff members. Eighteen of the twenty projects described 
were judged to be illustrative of the kinds of RDD&E 
activities likely to. occur within the context of education in 
the future. The twenty projects account for analyses 
around 298 project outputs and interviews with 134 
professional staff members. 

Volume V 

A METHODOLOGY FOR THE STUDY 
OF EDUCATIONAL RDD&E 

A detailed description of the most refined form of the data 
collection methodology developed within the Studies, 
directions to guide its use, and the decision rules needed for 
the volume to function as a users manual. The volume 
includes information on procedures used in site contact, 
site preparation, data reduction and analysis, and profile 
preparation. It also includes information on the category 
sets used in data reduction and the computerized data files 
that contain or provide access to all data collected in the 
Studies. 




Copies of any or all of these volumes 
may be obtained at cost from 
Teaching Research 
Monmouth, Oregon 97361 
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